Meta-Learning: Finding a Universal Starting Point

Task Distribution

Training tasks N 5

Inner steps K 4

Inner LR α 0.20

Meta LR β (Reptile step) 0.30

Live Readouts

Outer steps

Meta-loss (avg)

—

Held-out generalization gap

run test →

Legend

Training task minima

Held-out (test) task

Meta-init (shared start point)

Naive baseline (fixed origin)

How it works

Each task i is a quadratic loss bowl in 2D parameter space θ=(θx,θy):

L_i(θ) = a_i(θx − cx_i)² + b_i(θy − cy_i)²

Meta-learning searches for one shared starting point θ_meta that is only K gradient steps away from doing well on any task drawn from the distribution — not the best point for one task, but the best launchpad for all of them. This lab runs the Reptile algorithm, a first-order approximation of MAML:

for each outer step:
  sample a random training task i
  θ' = θ_meta
  repeat K times: θ' -= α · ∇L_i(θ')     (inner-loop adaptation)
  θ_meta += β · (θ' − θ_meta)            (outer-loop update)

The terrain height is the average loss across all training tasks at each (θx,θy) — a landscape shaped by every task at once, not any single one. The magenta marker is θ_meta drifting toward a region that is cheap to adapt from for every bowl simultaneously.

Test on Held-Out Task reveals the payoff: it runs the same K inner steps from θ_meta on a task the meta-learner never trained on, and compares the resulting loss to K steps from a fixed naive baseline at the origin. A large positive gap means the meta-learned initialization generalizes — it adapts to genuinely new tasks faster than starting from scratch, which is exactly what "learning to learn" means in practice (this is the same core idea behind MAML/Reptile-style few-shot adaptation used in real meta-learning research).

training tasks: 5 mode: idle

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.