This is a real feed-forward neural network (2 inputs → N hidden neurons → 1 output) learning the classic XOR problem, which is not linearly separable and requires a hidden layer to solve. Each sphere is a neuron; its brightness encodes its activation. Each glowing line is a weight; its color (orange = positive, blue = negative) and thickness encode the weight's sign and magnitude. A pulse travels along the network each step, showing the current forward pass for one of the 4 training samples.
z = W·x + b
a = sigmoid(z) = 1 / (1 + e^-z)
loss = mean((a_out - y)^2)
ΔW = -η · ∂loss/∂W (gradient descent)
- Learning rate η scales every backprop weight update; too high and the loss oscillates, too low and it crawls.
- Training speed runs multiple gradient-descent epochs per rendered frame.
- Hidden layer size changes network capacity and rebuilds the 3D graph.
- Pause/Reset freeze training or reinitialize all weights randomly and start over.
This same backpropagation algorithm, scaled up to billions of parameters, underlies modern deep learning systems used in computer vision and natural language processing.