Logistic Regression: The 80/20 Model
Logistic regression maps inputs through a sigmoid function to produce a probability between 0 and 1, the simplest form of binary classification.
Most classification problems in production don't need a neural network. They need logistic regression. It trains in seconds, requires minimal data, and the results are interpretable. Let's understand why.
The whole model fits in two moves: a and a squash. Multiply each feature by a learned , add a , push the result through a , and out comes a number between 0 and 1. Where a frontend if hands you a hard true/false, logistic regression hands you 0.83 and lets you decide what to do with it.
Learning Objectives
- ○Understand the sigmoid function and why it produces probabilities
- ○Implement logistic regression from scratch in TypeScript
- ○Interpret a decision boundary as a threshold on a confidence score
- ○Know when logistic regression is the right (and wrong) choice
The Sigmoid Turns Any Number Into Confidence
In frontend code, you write hard :
const isHighPriority = urgencyScore > 7; // boolean
But what if urgencyScore is 6.9? Or 7.1? A hard threshold gives you no nuance. The sigmoid function fixes this by mapping any number to a smooth probability between 0 and 1.
Frontend
if/else with confidence
const approved = score > 0.5 ? true : falseMachine Learning
Binary classification
const prob = sigmoid(weights.dot(features) + bias)// The sigmoid function: foundation of logistic regression
function sigmoid(x: number): number {
return 1 / (1 + Math.exp(-x));
}
// Large negative → close to 0
console.log(sigmoid(-10)); // 0.0000454
// Zero → exactly 0.5
console.log(sigmoid(0)); // 0.5
// Large positive → close to 1
console.log(sigmoid(10)); // 0.9999546
// In logistic regression, x = weighted sum of features + bias
function predict(features: number[], weights: number[], bias: number): number {
const z = features.reduce((sum, f, i) => sum + f * weights[i], 0) + bias;
return sigmoid(z);
}
// Example: predict if a user will convert
// features: [timeOnSite (min), pagesViewed, hasAccount]
const weights = [0.3, 0.5, 1.2];
const bias = -2.0;
const probConvert = predict([5, 8, 1], weights, bias);
console.log(probConvert); // 0.99, likely to convert (z = 4.7)
const probBounce = predict([0.5, 1, 0], weights, bias);
console.log(probBounce); // 0.21, likely to bounce (z = -1.35)The Decision Boundary
The decision boundary is just your threshold. By default it's 0.5: above that, classify as positive. But in production, you often tune this threshold:
function classify(
features: number[],
weights: number[],
bias: number,
threshold = 0.5
): { label: boolean; confidence: number } {
const confidence = predict(features, weights, bias);
return {
label: confidence >= threshold,
confidence,
};
}
// Medical screening? Lower threshold (catch more positives)
classify(features, weights, bias, 0.3);
// Spam filter? Higher threshold (fewer false positives)
classify(features, weights, bias, 0.8);Same move as tuning a media query breakpoint: you choose where the line falls based on what your use case punishes most.
Finding the Best Weights
Training logistic regression means finding the weights and bias that minimize prediction errors. The algorithm is called , the same concept that powers neural networks, just applied to a simpler model.
// Simplified training loop for logistic regression
function trainLogistic(
data: { features: number[]; label: number }[],
learningRate = 0.01,
epochs = 100
) {
const numFeatures = data[0].features.length;
const weights = new Array(numFeatures).fill(0);
let bias = 0;
for (let epoch = 0; epoch < epochs; epoch++) {
let totalLoss = 0;
for (const { features, label } of data) {
const pred = predict(features, weights, bias);
const error = pred - label;
// Update weights (gradient descent)
for (let i = 0; i < weights.length; i++) {
weights[i] -= learningRate * error * features[i];
}
bias -= learningRate * error;
// Binary cross-entropy loss
totalLoss += -(label * Math.log(pred) + (1 - label) * Math.log(1 - pred));
}
if (epoch % 10 === 0) {
console.log(`Epoch ${epoch}: loss = ${(totalLoss / data.length).toFixed(4)}`);
}
}
return { weights, bias };
}Challenge
Now implement logistic regression yourself.
Recall Prompt
What does the sigmoid function do, and why does logistic regression use it instead of a hard if/else?
Lesson Recap
What you learned
- ✓The sigmoid function transforms any number into a probability between 0 and 1, replacing a hard threshold with a confidence score
- ✓A decision boundary is a tunable threshold on that probability, adjustable for different cost trade-offs the same way you adjust a breakpoint
- ✓Logistic regression learns per-feature weights from data via gradient descent; start here before reaching for a neural network
The bridge
A frontend `if/else` with a hard threshold (`score > 0.5`) is a hand-coded rule; `Binary classification` with logistic regression learns that threshold and per-feature weights from data, so the decision boundary updates with retraining rather than code edits.
You can now
Implement logistic regression from scratch, interpret its probability output, and decide when it is the right model for a binary classification problem.