A small feed-forward network (2 inputs → hidden layer → 1 output) is trained live, in your browser, on the classic XOR dataset — the four points that are not linearly separable, which is exactly why a hidden layer is required to solve them.
Every animation frame the network runs several full-batch gradient-descent steps: a forward pass computes predictions, backpropagation computes the exact gradient of the loss with respect to every weight, and each weight is nudged a little downhill.
forward: h = f(x·W1 + b1)
ŷ = σ(h·W2 + b2)
loss: L = -mean[ y·log(ŷ) + (1-y)·log(1-ŷ) ]
update: W ← W - η · ∂L/∂W (η = learning rate)
- Learning rate — scales how far each gradient step moves the weights (η above).
- Hidden neurons — width of the hidden layer; rebuilds the network with fresh random weights.
- Training speed — how many gradient-descent steps run per rendered frame.
- Activation — the nonlinearity f() used inside the hidden layer: sigmoid, tanh, or ReLU.
- Sphere brightness shows each neuron's current activation on the training data; line color/opacity shows each connection weight's sign and magnitude.
This is the same core loop — forward pass, backprop, gradient step — that trains every modern deep learning model, from image classifiers to large language models, just scaled up from a handful of parameters to billions.