Home▸AI & Machine Learning▸Learned Optimizer Race: Meta-Learning to Descend a Loss Landscape

Learned Optimizer Race: Meta-Learning to Descend a Loss Landscape

Three optimizers race down the same ill-conditioned 3D loss bowl: plain SGD, momentum SGD, and a meta-learned per-coordinate adaptive rule discovered by learning-to-learn research — watch which one zig-zags and which one heads straight for the minimum.

AI & Machine Learning3DAdvanced60 FPS📱 Mobile-adapted⇄ 2D version

Meta-learning is not only about learning a good starting point (MAML, Reptile) or a good embedding space (Prototypical Networks) — it can also learn the optimizer itself. This simulator drops three update rules onto the same randomly-oriented, ill-conditioned quadratic loss bowl in 3D: plain SGD, momentum SGD, and a per-coordinate adaptive rule that mirrors what "learning to learn by gradient descent" research found a meta-trained optimizer converges to. Drag the condition-number slider to make the valley narrower and watch hand-designed SGD zig-zag while the learned rule drives straight down; resample a brand-new task to see the same learned rule generalize without retraining.

⚙ Under the hood

Three optimizers race down the same ill-conditioned 3D quadratic bowl — plain SGD, momentum SGD, and a meta-learned per-coordinate adaptive rule inspired by "learning to learn by gradient descent" research. Adjust the bowl's condition number and watch which rule zig-zags and which one heads straight for the minimum.

meta-learningoptimizationgradient descentlearning to learnneural networksloss landscape

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install