A small neural network trains live with real gradient descent — watch weighted connections, activations and the decision boundary evolve epoch by epoch.
How it works ▾
Each neuron computes a weighted sum of its inputs plus a bias, then squashes it with an activation function — the perceptron rule:
f(x) = Σ(wᵢ · xᵢ) + b
The network's error on the whole dataset is the loss L(w). Backpropagation computes the gradient ∇L(w) — how much each weight contributed to the error — and gradient descent nudges every weight a small step against it:
Δw = -η · ∇L(w)
Blue synapses carry positive weights, orange/pink synapses negative ones — thicker and brighter means a larger magnitude. The glowing pulses trace one forward pass through the layers. The floor plane renders the current decision boundary: colour = the class the network predicts for that point in input space, dots = the actual training samples.
- Dataset — XOR, concentric circles and an interleaved spiral are classic non-linearly-separable toy problems.
- Architecture — deeper/wider networks can fit more complex boundaries but train more slowly per epoch.
- Learning rate — too high overshoots and destabilises training, too low converges very slowly.