Home▸AI & Machine Learning▸Setting Initial Hyperparameter Values

🎯 Setting Initial Hyperparameter Values

Watch how different weight-initialization strategies place a model on a 3D loss landscape, and whether gradient descent finds the global minimum or gets stuck.

AI & Machine Learning3DAdvanced60 FPS

A 3D loss landscape where glowing particles, each representing a training run, start from positions determined by a chosen weight-initialization strategy and descend the gradient toward a marked global minimum — or get stuck along the way.

🔬 What It Demonstrates

Zero and oversized initial values tend to stall or overshoot, naive wide-random scatter is a gamble, while variance-scaled schemes like Xavier/Glorot and He keep runs in a band that reliably reaches the global minimum.

🎮 How to Use

Pick an init strategy, set the learning rate and landscape ruggedness, then watch the particles descend. Re-initialize to resample fresh starting points and compare how many reach the gold global-minimum marker.

💡 Did You Know?

Kaiming He's 2015 initialization scheme for ReLU networks uses roughly double the variance of Xavier/Glorot init, compensating for the fact that ReLU zeroes out half of its inputs on average.

⚙ Under the hood

Watch how different weight-initialization strategies place a model on a 3D loss landscape, and whether gradient descent finds the global minimum or gets stuck.

machine learningneural networksoptimizationhyperparametersgradient descentloss functionsThree.js

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install