Home▸AI & Machine Learning▸Learning Rate Scheduling Strategies Explained

📉 Learning Rate Scheduling Strategies Explained

Watch a ball descend a rugged 3D loss surface under different learning-rate schedules — step decay, cosine annealing, warmup, exponential decay — while a live curve plots the schedule and loss.

AI & Machine Learning3DAdvanced60 FPS

A ball performs live gradient descent across a rugged 3D loss surface, its step size at every moment set by the learning-rate schedule you choose, while a floating curve plots the schedule and the resulting loss over the run.

🔬 What It Demonstrates

How the shape of a learning-rate schedule — constant, step decay, cosine annealing, warmup, or exponential decay — changes the path an optimizer takes: whether it overshoots, oscillates, or settles smoothly into a minimum.

🎮 How to Use

Pick a schedule, base learning rate, total steps, and warmup length. Watch the ball descend the landscape in real time and compare the purple LR curve against the teal loss curve above the surface.

💡 Did You Know?

Cosine annealing with warm restarts (SGDR), introduced by Loshchilov & Hutter in 2016, is one of the most widely used schedules in modern deep learning, including large transformer training runs.

⚙ Under the hood

Watch a ball descend a rugged 3D loss surface under different learning-rate schedules — step decay, cosine annealing, warmup, exponential decay — while a live curve plots the schedule and loss.

machine learningoptimizationalgorithmsneural networksgradient descentlearning rateThree.js

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install