The simulation visualises activation distributions flowing through a multi-layer network in 3D, contrasting the drifting, spread-out distributions of unnormalized training with the stable, tightly clustered distributions produced when batch normalization is applied.
Toggle batch normalization on or off to compare training behaviour, use the speed slider to control how fast training steps play out, and use play/pause and rebuild to restart the network with fresh random weights.
Normalization toggle, speed slider, play/pause, rebuild
The original 2015 batch normalization paper allowed one image classification model to match its previous accuracy using 14 times fewer training steps, while also beating the original network's best accuracy outright.
The simulation visualises activation distributions flowing through a multi-layer network in 3D, contrasting the drifting, spread-out distributions of unnormalized training with the stable, tightly clustered distributions produced when batch normalization is applied.
The simulation visualises activation distributions flowing through a multi-layer network in 3D, contrasting the drifting, spread-out distributions of unnormalized training with the stable, tightly clustered distributions produced when batch normalization is applied.
Toggle batch normalization on or off to compare training behaviour, use the speed slider to control how fast training steps play out, and use play/pause and rebuild to restart the network with fresh random weights.
The original 2015 batch normalization paper allowed one image classification model to match its previous accuracy using 14 times fewer training steps, while also beating the original network's best accuracy outright.