Double Descent: When More Parameters Beat the Bias-Variance Tradeoff

This simulator runs a genuine random-features ridge regression in your browser — not a canned animation. On the left, a fixed set of noisy training points is fit by a model whose capacity (number of random Fourier features) you control; on the right, the actual computed train and test error are plotted against that capacity as a 3D curve. Slide capacity up and classical statistics shows up first: error falls, then rises as the model starts to overfit. But keep going past the interpolation threshold — where the number of features equals the number of training samples — and, after a sharp spike, test error falls a second time. That non-monotonic "double descent" shape, discovered in modern over-parameterized machine learning, is why today's neural networks with vastly more weights than training examples often generalize better, not worse.