Presets
Playback
Data
Network
Training
Probe
Layers
Model check
The science
The network
A multilayer perceptron computes layer by layer a(l) = f(W(l) a(l−1) + b(l)). Hidden layers use the activation you pick (tanh, ReLU or sigmoid); the single output neuron uses a sigmoid p = 1/(1 + e−z) and is read as the probability that the point is red. Weights are initialised with Glorot/He scaling from a seeded random generator.
Loss and backpropagation
The loss is the mean binary cross-entropy L = −(1/N) Σ [y ln p + (1 − y) ln(1 − p)] + (λ/2) Σ w² (biases are not decayed). For a sigmoid output with cross-entropy the output error is simply δ(L) = p − y; it is passed backwards with δ(l) = (W(l+1))T δ(l+1) ⊙ f′(z(l)), and the gradients are ∂L/∂W(l) = δ(l) (a(l−1))T, ∂L/∂b(l) = δ(l). This is the code that runs in the page, written by hand with no library.
Optimisers
Plain SGD: w ← w − η g. Momentum: v ← 0.9 v − η g; w ← w + v. Adam keeps running estimates of the first and second moments of the gradient, w ← w − η m̂ / (√v̂ + 10−8) with β1 = 0.9, β2 = 0.999. Mini-batches are drawn without replacement from a shuffled epoch.
What to look for
A single neuron can only draw a straight line, so XOR needs at least two hidden units, and the spiral needs several layers. When the training loss keeps falling while the test loss rises, the network is memorising noise (overfitting); L2 decay λ shrinks the weights and smooths the boundary. The dashed line at ln 2 = 0.693 is the loss of a network that always answers 50 %.
Honest simplifications
Tiny two-input problems with synthetic data, a single output unit and full-precision (64-bit) arithmetic in the browser; no batch normalisation, dropout or learning-rate schedule; test accuracy is measured on 200 fresh points from the same generator, so it is an estimate with a few percent of noise. Real deep networks differ mainly in scale, convolution/attention structure and the data pipeline, not in the backpropagation rule.
Checks run in the page
(1) Backpropagation against centred finite differences for every parameter of the current network (the scatter plot shows analytic against numerical gradients on the diagonal). (2) Swapping two hidden units together with their weights leaves the output unchanged. (3) With an all-zero output layer the loss equals ln 2 exactly. (4) A 2-4-1 tanh network trained with Adam learns XOR on four points to 100 % accuracy.