Tensors as Linear Algebra
Tensors are elements of vector spaces with transformation rules, not just arrays of numbers.
You've been creating tensors and running operations on them since Module 1. But there's a deeper structure here that most tutorials skip. Tensors aren't just arrays. They're elements of vector spaces, and the operations you've been using are linear maps. Understanding this changes how you think about every layer in a neural network.
Learning Objectives
- ○Understand vector spaces and why tensors live inside them
- ○Identify basis vectors and linear independence in practical terms
- ○Perform matrix multiplication as composition of linear maps
- ○Connect CSS transform matrices to ML weight matrices
- ○Compute the rank of a matrix and understand what it means for data
From Arrays to Vector Spaces
In frontend development, you've used CSS transforms without thinking twice:
transform: matrix(1, 0, 0, 1, 50, 100);
That 2D transform matrix lives in a vector space. Every CSS transform you've ever written is a linear map: it takes a coordinate, produces a new coordinate, and respects addition and scaling. ML matrices work identically.
Frontend
CSS Transform
transform: matrix(a, b, c, d, tx, ty)Machine Learning
Linear Map
const result = tf.matMul(transformMatrix, vector)A vector space is a set of objects (vectors) where you can add any two together and multiply any one by a scalar, and the results stay in the set. JavaScript arrays of fixed length form a vector space. You can add [1, 2, 3] and [4, 5, 6] element-wise to get [5, 7, 9], and that result is still a 3-element array.
Basis Vectors and Coordinates
Every vector in a space can be written as a combination of basis vectors. In 2D, the standard basis is:
- e1 = [1, 0], the x-axis direction
- e2 = [0, 1], the y-axis direction
The point [3, 7] is really 3 * e1 + 7 * e2. CSS coordinates work the same way: the browser defines a basis (x-right, y-down), and every position is a linear combination of those directions.
import * as tf from '@tensorflow/tfjs';
// Standard basis vectors in 2D
const e1 = tf.tensor1d([1, 0]);
const e2 = tf.tensor1d([0, 1]);
// Any 2D vector is a linear combination of basis vectors
const point = tf.add(e1.mul(3), e2.mul(7));
console.log(await point.array()); // [3, 7]
// A weight matrix transforms basis vectors to new positions
// (exactly what a CSS transform matrix does). Under matMul(W, point),
// the COLUMNS of W are where the basis vectors land:
const W = tf.tensor2d([
[2, -1],
[1, 3]
]);
// column [2, 1] is where e1 lands; column [-1, 3] is where e2 lands
// Transform our point
const transformed = tf.matMul(W, point.reshape([2, 1]));
console.log(await transformed.array());
// [[-1], [24]]: the point [3,7] mapped through the linear transformationLinear Independence and Rank
Vectors are linearly independent if none can be written as a combination of the others. The of a matrix tells you how many independent directions it preserves. If a weight matrix has low rank, it's collapsing your data into fewer . Information is being lost.
import * as tf from '@tensorflow/tfjs';
// Full-rank matrix: preserves both dimensions
const fullRank = tf.tensor2d([
[1, 0],
[0, 1]
]);
// Rank-1 matrix: collapses 2D data onto a line
const rank1 = tf.tensor2d([
[1, 2],
[2, 4] // Row 2 = 2 * Row 1, not independent!
]);
// Apply rank-1 matrix to different points
const points = tf.tensor2d([[1, 0], [0, 1], [1, 1]]);
const collapsed = tf.matMul(points, rank1.transpose());
console.log(await collapsed.array());
// All outputs land on the same line: dimension reduced
// In ML, this is why weight initialization matters:
// a rank-deficient weight matrix can't learn distinct features
// Reveal the rank with a QR decomposition: tf.linalg.qr rewrites the
// matrix as Q (a rotation) times R (upper triangular). A zero on R's
// diagonal marks a column that added no new direction.
const [q, r] = tf.linalg.qr(rank1);
console.log(await r.array());
// Second diagonal entry of R is 0: rank 1Matrix Multiplication as Composition
When you chain CSS transforms, the browser multiplies the matrices together. Neural network work the same way: each layer's weight matrix is a linear map, and layers composes those maps.
import * as tf from '@tensorflow/tfjs';
// Two transformations (like two neural network layers)
const rotate = tf.tensor2d([
[0, -1],
[1, 0] // 90-degree rotation
]);
const scale = tf.tensor2d([
[2, 0],
[0, 3] // Scale x by 2, y by 3
]);
// Composing: scale THEN rotate = single matrix
const combined = tf.matMul(rotate, scale);
console.log(await combined.array());
// [[0, -3], [2, 0]]
// This is why deep networks are powerful:
// each layer adds a new linear map (plus nonlinearity)
// The composition can represent incredibly complex transforms
// Without activation functions, N layers collapse to 1 matrix:
const layer1 = tf.randomNormal([4, 3]);
const layer2 = tf.randomNormal([3, 4]);
const collapsed = tf.matMul(layer2, layer1);
// collapsed is 3x3: two layers = one linear map
// This is why we NEED nonlinear activations between layersChallenge
Put your linear algebra knowledge to work by implementing core vector space operations.
Recall Prompt
If you stack two linear layers with no activation function between them, how many effective linear maps do you get?
Lesson Recap
What you learned
- ✓A matrix applied to a vector is a linear map: it scales and rotates the space while respecting addition and scalar multiplication.
- ✓Matrix rank measures how many independent directions a transformation preserves; a rank-deficient matrix collapses data into fewer dimensions and loses information.
- ✓Stacking linear layers without activations is equivalent to a single matrix multiply, because composing linear maps produces another linear map.
The bridge
Just as chaining CSS `transform: matrix()` calls multiplies the matrices together, stacking neural network weight matrices composes linear maps into one combined transformation.
You can now
Explain why neural networks need nonlinear activations and how matrix rank affects what a layer can represent.