🧠 How Hyperparameters Affect Model Performance

About this simulation

The bumpy terrain is a stylised loss landscape — every point on the floor is a possible pair of model weights, and the height is the training loss at that point. A ball marks the model's current weights and rolls downhill using gradient descent, exactly like a neural network training step repeated over and over. The glowing trail shows the path it took to get there.

How it works

• Learning rate — how big a step the optimizer takes down the slope each update. Too small crawls forever; too large overshoots the minimum and can bounce out of the valley entirely (divergence).
• Momentum — carries part of the previous step's velocity forward, helping the ball power through shallow bumps but making it overshoot narrow valleys if set too high.
• Batch size — small batches estimate the gradient from fewer examples, injecting visible jitter into every step; large batches give smoother, more confident steps but move less randomly.
• L2 regularization — penalizes complexity, which here smooths away sharp local bumps and pulls the whole landscape toward a single simple bowl, at the cost of flattening some real detail.

Controls

• Learning rate / Momentum / Batch size / Regularization sliders — change the optimizer and landscape live.
• ↺ Reset run — respawns the ball at a fresh random start point on the current landscape.
• Drag to rotate the view, scroll to zoom.

Did you know?

Real loss landscapes for deep networks live in millions of dimensions, but the same failure modes shown here — overshoot from a too-high learning rate, noisy wandering from small batches, sluggish crawling from a too-low learning rate — are exactly what practitioners watch for in real training curves.