Feature Scaling Lab 2D: Gradient Descent Race Track

Dataset

Data points 80

Outlier strength 1.0×

Shared learning rate η 0.010

Focus lane

Training

GD iterations

Focus loss (MSE)

—

Focus mean

—

Focus std / spread

—

How it works

Four lanes hold the same two-feature dataset — income (≈20k–200k) and age (≈20–70) — transformed four different ways, then each fits a linear model with plain batch gradient descent using one shared learning rate η:

Raw:        x' = x
StandardScaler: x' = (x − μ) / σ
MinMaxScaler:   x' = (x − min) / (max − min)
RobustScaler:   x' = (x − median) / (Q75 − Q25)

update:  w ← w − η · ∇MSE(w),   MSE = mean((w·x' + b − y)²)

Income and age live on wildly different scales, so the loss surface for the Raw lane is a long, narrow ellipse — its gradient with respect to income is enormous compared to age's, so a learning rate that is safe for one variable overshoots wildly on the other and the fit diverges (rising tower, flagged red). Scaling both features onto comparable ranges makes the loss surface roughly circular, so the same η converges smoothly.

Outlier strength — injects one extreme income point (the white ring on each lane's scatter). MinMaxScaler stretches its whole range to fit it, crushing every normal point toward 0. StandardScaler inflates its mean/variance, so it degrades too, just less violently. RobustScaler uses median/IQR and barely notices.
Learning rate — shared across all four lanes on purpose: it is exactly the parameter a real training run only gets to set once.
Point-cloud spread on each lane's track is drawn to scale within that lane (both axes divided by the lane's own largest magnitude) — a squashed line means one feature is dominating the other numerically.
The vertical bar behind each lane is a top-down "loss tower": its fill height tracks log(1+loss), exactly like the rising ball in the 3D version, and flashes red on divergence.

Drag to pan across the four lanes, scroll or pinch to zoom in on any one track.

Raw (unscaled) StandardScaler MinMaxScaler RobustScaler

drag to pan · scroll to zoom