⛰️ Nesterov vs Heavy-Ball Momentum on an Ill-Conditioned Quadratic Valley: Condition Number and Convergence Rate - 3D Blender Lab
Roll gradient descent, heavy-ball momentum and Nesterov acceleration down the same narrow quadratic valley and see why the condition number sets the speed: exact two-term recurrences, spectral convergence rates, optimal momentum and the overshoot of too much beta.
⚙ Under the hood
Roll gradient descent, heavy-ball momentum and Nesterov acceleration down the same narrow quadratic valley and see why the condition number sets the speed: exact two-term recurrences, spectral convergence rates, optimal momentum and the overshoot of too much beta.
Nesterov momentumheavy ballgradient descentcondition numberconvergence rateoptimizationquadraticmachine learning
3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install