Weight Initialization & Vanishing Activations Lab

This simulation trains a real, configurable feedforward neural network — 2 to 6 hidden layers, sigmoid/tanh/ReLU activations — on a 2D circle-classification task using genuine backpropagation, entirely client-side. Every layer of the network is rendered in 3D as a slab of bars whose height and colour encode the standard deviation of each neuron's activation across the training batch, making the classic deep-learning failure modes visible as they happen: symmetric zero initialization that never breaks symmetry, large random weights that saturate sigmoid/tanh and vanish by the final hidden layer, and the Xavier/Glorot and He schemes that keep signal variance roughly constant through depth. Pick an initialization, an activation function and a network depth, then watch the loss curve, accuracy and the deep-layer activation spread evolve — or fail to — as training proceeds.