Regularization adds a penalty term to a model's loss function so it prefers smaller, simpler weights instead of chasing every wiggle in the training data. The left shape is a 3D loss "bowl" for two weights (w1, w2); the purple point marks where the loss plus the penalty is minimized. The right side shows dropout, a different regularizer that randomly disables neurons during training so the network can't over-rely on any single one.
Lasso's corner-hugging behaviour was formalized by Robert Tibshirani in 1996; because the L1 ball has vertices on the coordinate axes, the loss contours are statistically most likely to first touch the constraint region exactly at those vertices — which is precisely where one or more weights equal zero.
A two-part 3D lab: a loss "bowl" shows how L1, L2 and elastic-net penalties shrink and sparsify a model's weights as lambda grows, while a linked neural network shows dropout randomly thinning hidden neurons.
The purple marker is the closed-form regularized solution for two weights; watch it slide from the grey unregularized optimum toward the origin as lambda increases, and land exactly on an axis under L1 — that's sparsity.
Pick a penalty type, drag Lambda to change regularization strength, and set the Dropout rate to see more or fewer hidden neurons switched off each training step. Resample the dropout mask on demand.
Because the L1 constraint region has sharp corners on the weight axes, gradient contours are far more likely to first touch it exactly at a corner — which is why Lasso regression tends to zero out entire features.