Training a neural network means walking downhill across a high-dimensional loss landscape in search of a minimum. This simulation compresses that landscape to two weight dimensions so you can see, in 3D, exactly what hyperparameters actually do to the optimizer's path: a marker rolls across a bumpy surface built from several overlapping basins (a stand-in for a real non-convex loss surface with a good minimum and misleading local minima).
Real loss landscapes for deep networks live in millions of dimensions, but empirically most directions around a well-trained minimum are flat — the effective "bumpiness" that traps optimizers is concentrated in only a handful of directions, which is part of why modest amounts of momentum and noise so reliably help in practice.
A glowing marker trains in real time across a 3D loss landscape carved from overlapping basins — watch how learning rate, momentum and optimizer choice decide whether it settles into the deep global minimum or gets stuck in a shallow local one.
The marker's height is the current loss; its purple trail is the optimizer's trajectory. Vanilla SGD, momentum SGD and a simplified Adam step are all computed live from the same underlying gradient each epoch.
Pick an optimizer, then tune learning rate, momentum and batch-size noise. Watch epoch, loss, gradient norm and best loss update live, and restart training from a fresh random point to compare settings.
Adam adapts its effective step size per-parameter using running estimates of the gradient's mean and variance — which is why it often needs a much larger nominal learning rate than plain SGD to move at a comparable pace.