Home▸AI & Machine Learning▸Learned Optimizer Race: 2D Contour Map of a Loss Landscape

Learned Optimizer Race: 2D Contour View

Top-down contour-map version of the optimizer race: plain SGD, momentum SGD and a meta-learned per-coordinate adaptive rule descend the same ill-conditioned quadratic bowl, with live loss curves, a gradient-field overlay and pan/zoom over the loss landscape.

AI & Machine Learning2DAdvanced60 FPS📱 Mobile-adapted⇄ 3D version

Details

CategoryAI & Machine Learning

RenderingCanvas 2D

DifficultyAdvanced

Target FPS60

ControlsDrag the map to pan and scroll to zoom over the contour view. Drag the condition-number and learning-rate sliders to reshape the bowl and step size, tune the momentum β and adaptive-decay ρ constants directly, adjust step speed and trail length, toggle the gradient-descent field overlay, click New Task to resample the bowl's rotation and a fresh random start point, and Restart to replay the same task from scratch.

meta-learningoptimizationgradient descentlearning to learnneural networksloss landscapecontour map

Related simulations

Batch Size HyperparametersA 3D loss-landscape simulator showing how batch size controls gradient noise, convergence…

Gradient-Based Methods for Hyperparameter OptimizationA 3D validation-loss surface over two hyperparameters where a marker rolls downhill by…

Gradient Descent and Backpropagation Training LabAn interactive 3D loss landscape where a marker descends toward a minimum, letting you… ⊞ All 1161 in AI & Machine Learning →

Meta-learning is not only about learning a good starting point (MAML, Reptile) or a good embedding space (Prototypical Networks) — it can also learn the optimizer itself. This top-down contour-map simulator drops three update rules onto the same randomly-oriented, ill-conditioned quadratic loss bowl: plain SGD, momentum SGD, and a per-coordinate adaptive rule that mirrors what "learning to learn by gradient descent" research found a meta-trained optimizer converges to. Drag the condition-number slider to make the valley narrower and watch hand-designed SGD zig-zag while the learned rule drives straight down; pan and zoom over the contour map, toggle the gradient-descent field, and watch the live loss-vs-step chart update in real time.

⚙ Under the hood

Top-down contour-map version of the optimizer race: plain SGD, momentum SGD, and a meta-learned per-coordinate adaptive rule descend the same ill-conditioned quadratic bowl. Pan and zoom over the contour map, toggle a live gradient-descent field, and tune the momentum and adaptive-decay constants that the 3D version keeps hidden.

meta-learningoptimizationgradient descentlearning to learnneural networksloss landscapecontour map

2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install