The task is an ill-conditioned quadratic bowl — the standard stress test used in learning-to-learn research. This 2D version renders it as a top-down contour map instead of a 3D surface, which is closer to how the optimizer actually "sees" the landscape: everything happens in the (x,y) plane, and height was only ever a visualization of loss.
f(x,y) = 0.5·(λ₁u² + λ₂v²)
u = x·cosθ + y·sinθ
v = −x·sinθ + y·cosθ
κ = λ₂ / λ₁ (condition number)
A high κ means the bowl is a steep, narrow valley in one direction and almost flat in the other — exactly the geometry that makes fixed-step gradient descent zig-zag. Each optimizer updates its own position θ=(x,y) from the same gradient ∇f(θ) (verified numerically against a finite-difference gradient before shipping):
SGD: θ ← θ − α∇f(θ)
Momentum: v ← β·v − α∇f(θ)
θ ← θ + v
Learned: s ← ρ·s + (1−ρ)·∇f(θ)²
θ ← θ − α·∇f(θ) / (√s + ε)
The "Learned Optimizer" rule is a per-coordinate adaptive step: it divides the gradient by a running estimate of its own recent magnitude, so steep directions get squashed and flat directions get boosted automatically. This mirrors the behaviour that "Learning to Learn by Gradient Descent by Gradient Descent" (Andrychowicz et al., 2016) found an LSTM meta-optimizer converges to after being meta-trained across many random quadratic tasks — the network's outer loop tunes a general update rule, and each new bowl is the inner loop the rule must solve without any further tuning.
- κ slider — reshapes the bowl's anisotropy; higher κ is a harder task for hand-designed SGD.
- α slider — the shared base learning rate every optimizer's rule is built on.
- β / ρ sliders — expose the momentum and adaptive-decay constants normally hidden inside the update rules.
- Gradient field — draws −∇f at a grid of points so you can see the descent direction everywhere, not just along the three trails.
- Drag the map to pan, scroll to zoom. New Task resamples the bowl's rotation θ and a fresh random start point.