Layer Normalization: Per-Sample Feature Scaling (2D)
A 2D-canvas companion to the 3D layer-normalization sim: watch a grid of samples get re-centered to mean 0 / variance 1 across their own features, drag gamma/beta, and flip the normalization axis to see exactly how Batch Normalization differs.
This 2D-canvas companion to the 3D layer-normalization sim renders the same sample x feature grid as flat bar cells instead of 3D bars: the dim top band is each sample's raw activations, the bright bottom band is the same values after subtracting a mean, dividing by a standard deviation, and applying the learnable γ (scale) and β (shift). What makes this version different is the axis toggle — flip between "Per-Sample (LayerNorm)", which normalizes every row against its own features exactly as inside a Transformer block, and "Per-Feature (BatchNorm)", which normalizes every column against the other samples in the batch instead. Shrink the batch in BatchNorm mode and watch its normalized values become noisy and unstable, while LayerNorm mode stays rock-solid regardless of batch size — the single distinction that explains why Transformers and RNNs use LayerNorm while CNNs lean on BatchNorm.
A 2D-canvas companion to the 3D layer-normalization sim, with a toggle that flips the normalization axis between per-sample (LayerNorm) and per-feature (BatchNorm) to show why LayerNorm is batch-size independent.
2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install