Home▸AI & Machine Learning▸Learning Rate Hyperparameters

📉 Learning Rate Hyperparameters

Watch a gradient-descent 'ball' roll across a 3D loss surface and see how learning rate, momentum, and schedule choices change convergence, oscillation, and divergence.

AI & Machine Learning3DAdvanced60 FPS

A ball rolls across a 3D loss surface under real gradient descent with momentum, letting you feel how learning rate and schedule choices trade off convergence speed against oscillation and divergence.

🔬 What It Demonstrates

Each step computes the local gradient and updates position by learning rate × slope plus momentum-carried velocity — the same update rule used in SGD, so overshoot, oscillation, and stalling behave exactly as they do in real training.

🎮 How to Use

Set the learning rate and momentum, pick a schedule and a loss surface, then watch the loss chart. Push the learning rate high on the ravine surface to see divergence; keep it low on the bumpy surface to see it stall in a local minimum.

💡 Did You Know?

Cosine annealing and step decay were popularized because a large learning rate helps escape shallow local minima early in training, while a small one late in training lets the optimizer settle precisely into a deep minimum.

⚙ Under the hood

Watch a gradient-descent 'ball' roll across a 3D loss surface and see how learning rate, momentum, and schedule choices change convergence, oscillation, and divergence.

machine learningoptimizationgradient descenthyperparametersalgorithmsneural networksThree.js

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install