The loss is the exact same quadratic bowl as the 3D version — f(x,y) = ½(κ·x² + y²), gradient ∇f = (κ·x, y) — but drawn here as a true 2D contour map: each ring is an analytic level curve of f, a real ellipse with semi-axes a=√(2c/κ) and b=√(2c) for level c, not a projected 3D surface. Faint arrows sample the gradient field itself; three walkers descend it with three different update rules, plotted live on the right as log₁₀(loss) versus step count so you can read off the actual convergence rate, not just eyeball a trajectory.
Plain GD: θ ← θ − η∇f(θ)
Momentum: v ← β·v − η∇f(θ)
θ ← θ + v
Nesterov: v ← β·v − η∇f(θ + β·v) ← gradient evaluated at the
θ ← θ + v look-ahead point, not θ itself
Plain GD is stable only while η < 2/κ, so a large κ forces a tiny step in the steep x-direction — and that same tiny step crawls painfully slowly down the shallow y-direction, visible on the contour map as tight zig-zagging between opposite walls of a narrow ellipse. Classical momentum accumulates velocity and reaches the shallow floor faster but overshoots and rings, because it evaluates the gradient at the point it is currently standing on. Nesterov evaluates the gradient at θ + β·v — where momentum is about to carry it — damping the overshoot before it happens. On the convergence chart this shows up as a straighter, steeper descending line: NAG's loss falls at the accelerated O(1/k²) rate instead of GD's much slower linear-in-log crawl.
- κ slider — elongates the valley (thinner ellipses); higher κ makes plain GD's zig-zag dramatically worse while barely slowing Nesterov's chart line.
- η slider — step size shared by all three optimizers, for a fair race.
- β slider — momentum coefficient; push it toward 0.98 to see classical momentum's chart line start oscillating (loss bouncing up and down) while Nesterov's stays smooth.