Data Normalization

Commander Mirza

Before the deep scan runs, normalize every sensor feed. One scale, no exceptions.

Cut to: Briefing

Ensign Demir

The thermal channel reads in the thirties, signal strength lives under one. If we feed both in raw, the model's going to think temperature is the whole story.

Cut to: Briefing

Dr. Farah

With comparable starting weights, temperature has the larger numerical contribution. Scaling helps us fit the model; it does not tell us which sensor matters.

Cut to: Briefing

Why Scale Features

Min-max normalization is the arithmetic under a scroll progress bar. scrollY / (docHeight - viewportHeight) takes a value in pixels and maps it to 0-1 so the bar can use it. Subtract the minimum, divide by the range: that's the entire formula, and it does the same job on a sensor channel that it does on a scroll position.

For this neural network, scaling can help training. The temperature channel runs 19 to 35. Signal strength runs 0.1 to 0.9. A typical temperature value is near 25 and a typical signal strength near 0.5, so temperature's numbers arrive about fifty times larger. With comparable weights, temperature contributes more to the weighted sum and can produce larger gradients. Scaling can make optimization easier; it does not decide which feature is informative. A trained model can still assign different weights to the features.

Two names in the code below. The standard deviation is the typical distance between a value and its column's mean, so dividing by it measures a feature in its own units of spread: a z-score of 1 sits one typical distance above the mean, whatever the raw units were. tf.moments hands back the variance instead, that distance squared and averaged, so the standard deviation is variance.sqrt(). The averaging divides by the number of rows, n, which makes both statistics the population version. The usual unbiased sample variance divides by n minus 1. Its square root is larger than the population standard deviation when n > 1 and variance is nonzero; that square root is not itself an unbiased estimator of standard deviation.

Challenge

Normalized data exists to be fed to a model, so this challenge ends at one. Farah hands you one frozen neuron off her scratch pad, weights fixed: two features in, one verdict out. That is enough to show why the saved statistics matter. Feed it a reading normalized with different stats and it answers a different question.

Normalize the Archimedes sensor data using z-score normalization, then run every reading through the preview neuron.