⛰️ Nesterov vs Heavy-Ball Momentum on an Ill-Conditioned Quadratic Valley: Condition Number and Convergence Rate (2D)
Roll gradient descent, heavy-ball momentum and Nesterov acceleration down the same narrow quadratic valley and see why the condition number sets the speed: exact two-term recurrences, spectral convergence rates, optimal momentum and the overshoot of too much beta.
⚙ Under the hood
Roll gradient descent, heavy-ball momentum and Nesterov acceleration down the same narrow quadratic valley and see why the condition number sets the speed: exact two-term recurrences, spectral convergence rates, optimal momentum and the overshoot of too much beta.
Nesterov momentumheavy ballgradient descentcondition numberconvergence rateoptimizationquadraticmachine learning
2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install