Data Normalization

import * as tf from '@tensorflow/tfjs';

// Sensor data: temperature in °C, signal strength as a 0-1 fraction
const temperature = tf.tensor([20, 22, 19, 35, 28]);  // range: 19-35
const signal = tf.tensor([0.1, 0.8, 0.3, 0.9, 0.5]);  // range: 0.1-0.9

// Z-score normalization: center at 0, spread to ~1.
// TFJS has no tensor.std(); moments() gives mean and variance together.
function zNormalize(tensor: tf.Tensor) {
return tf.tidy(() => {
const { mean, variance } = tf.moments(tensor);
const std = variance.sqrt();
// where(condition, yes, no) selects per feature. A constant feature
// has std 0: use a divisor of 1, so its training values become 0.
const divisor = tf.where(std.equal(0), tf.onesLike(std), std);
return tensor.sub(mean).div(divisor);
});
}

// One argument you'll need the moment features live side by side in a
// 2D batch: the AXIS. tf.moments(batch, 0) reduces DOWN axis 0 (down
// the rows), producing one mean and one variance PER COLUMN, so each
// feature is scaled by its own statistics. Without the axis you get a
// single global mean over every number in the batch, which stirs
// temperature and signal strength into one meaningless average.
const batch = tf.stack([temperature, signal], 1);  // [5, 2]: 5 readings x 2 features
const colMeans = tf.tidy(() => tf.moments(batch, 0).mean);
// colMeans: [24.8, 0.52], one per column, not one for the whole batch

const tempNorm = zNormalize(temperature);
// [-0.80, -0.47, -0.97, 1.71, 0.54]: centered around 0

const signalNorm = zNormalize(signal);
// [-1.40, 0.94, -0.73, 1.27, -0.07]: same scale!

// Min-max normalization: squeeze to [0, 1] range
// (the scroll-progress formula, applied to sensors)
function minMaxNormalize(tensor: tf.Tensor) {
return tf.tidy(() => {
const min = tensor.min();
const max = tensor.max();
const range = max.sub(min);
const divisor = tf.where(range.equal(0), tf.onesLike(range), range);
return tensor.sub(min).div(divisor);
});
}

const tempMinMax = minMaxNormalize(temperature);
// [0.06, 0.19, 0.0, 1.0, 0.56]: all between 0 and 1

A constant training feature has zero variance and zero range, so ordinary division produces invalid values. The examples use a divisor of 1 only for those zero scales. Constant training values then map to 0. Reuse that rule at inference: a changed value becomes its difference from the saved mean or minimum. That change needs investigation because training showed no variation in that feature.

One practical wrinkle that bites in production: whatever mean and std you compute here, you must save. Inference data has to be normalized with the training statistics, not its own; otherwise the same reading maps to different numbers on different days, and your model quietly answers a different question each time.

Which rows you compute them from matters too. Lesson 4 kept the validation rows out of training, and the statistics follow the same rule. Compute mean and std on the training rows only, then apply them to validation and to every later reading. Fitting them on training plus validation lets held-out information affect the model’s inputs.

When to Use Which

Z-score is a common choice for neural network inputs. It centers each nonconstant training feature at mean 0 and standard deviation 1. Extreme values still affect both statistics; check outliers rather than assuming this scaler removes their influence.

Min-max maps the fitted minimum and maximum to 0 and 1. Known physical bounds can supply those limits, as with pixel values. Fitted bounds are also usable, but later values can fall outside 0-1. One extreme value can compress the rest into a narrow range.

What the Preview Neuron Computes

The exercise ends at a model, so here is all of it. tf.sequential() holds an ordered list of layers, and here the list is one entry long: a single dense unit that reads two features. Farah's frozen numbers are one weight per feature, 0.9 for temperature and -1.4 for signal strength, plus one bias of -0.3.

// One neuron, built and frozen: two features in, one verdict out.
const neuron = tf.sequential();
neuron.add(tf.layers.dense({ units: 1, activation: 'sigmoid', inputShape: [2] }));

// setWeights is positional: the kernel first, one weight per input feature
// arranged as a column, then the bias, one number per unit. Farah's numbers
// arrive as plain JS arrays, so each becomes a tensor on the way in.
tf.tidy(() => neuron.setWeights([tf.tensor2d([[0.9], [-1.4]]), tf.tensor1d([-0.3])]));

// predict reads a batch: [n, 2] rows in, [n, 1] verdicts out. Stack the two
// z-scored columns from above the same way the raw batch was stacked.
const normalizedBatch = tf.stack([tempNorm, signalNorm], 1);  // [5, 2]
const verdicts = neuron.predict(normalizedBatch) as tf.Tensor;
console.log(await verdicts.array());
tf.dispose([temperature, signal, batch, colMeans, tempNorm, signalNorm,
tempMinMax, normalizedBatch, verdicts]);
neuron.dispose();

The neuron multiplies each normalized feature by its weight and adds the bias. It passes the total through sigmoid, 1 / (1 + Math.exp(-x)), which squashes any number into the range 0 to 1.

The numbers below come from the exercise's four-reading batch, not the five readings above: its per-column mean is [23, 0.5] and its std is [2.24, 0.224]. Against those stats, reading [20, 0.2] z-scores to [-1.34, -1.34]. The total is 0.9 * -1.34 + -1.4 * -1.34 - 0.3, or 0.37, and sigmoid of that is 0.59. Reading [22, 0.8] z-scores to [-0.45, 1.34], total -2.58, verdict 0.07. Both are worth checking on a calculator. predict() runs that arithmetic for a whole batch, one row in, one score out. It always wants a batch, so a single reading goes in as one row of a [1, 2] tensor rather than the bare [2] you are holding.

Now run those same four readings through the same neuron raw, without normalizing, and every verdict comes back 1.000000. The temperature term alone gets there: 0.9 * 20 is 18, and sigmoid is flat at the top long before signal strength contributes its -1.4 * 0.2. Normalized, the same neuron answers 0.59, 0.07, 0.67 and 0.57, and the lowest of the four belongs to [22, 0.8], the reading with the strongest signal. Same weights, same bias, both times. Only the scales changed.

Module 2 builds these layers and module 3 trains their weights. Tonight the layer is a fixed function you feed. Its score is not a calibrated probability. A familiar reading near the decision boundary can score near 0.5, while an unfamiliar reading can receive a confident score. Neither value tells you whether the model has seen that pattern before.