This simulation trains a small feed-forward neural network — two hidden layers on top of a 2D input — on a binary classification problem entirely in your browser, so you can watch backpropagation and gradient descent reshape the decision boundary in real time. Choose a dataset, pick an optimiser, and press Train to see the network carve out a region for each class while the loss curve tracks both training and validation error.
Every training step samples a mini-batch, runs it through two ReLU hidden layers and a sigmoid output, computes the binary cross-entropy loss L = −Σ[y·log ŷ + (1−y)·log(1−ŷ)], and backpropagates the gradient to update every weight. The left canvas colours each pixel of input space blue or red based on the network's live prediction, so the evolving decision boundary is visible pixel by pixel, while the right canvas plots training loss (blue) against validation loss (red) so you can spot overfitting the moment the two curves diverge.
Pick a Dataset (XOR, Circles, Moons or Spiral) to change the shape the network must learn to separate, and adjust Hidden units per layer, Learning rate η, Batch size and L2 regularisation λ before pressing Train. Switch Optimiser between SGD, Momentum and Adam to compare convergence speed and stability — Adam uses bias-corrected first and second moment estimates (m̂ and v̂) and typically converges fastest, while plain SGD is the most sensitive to the learning rate slider.
The XOR problem was famously used by Minsky and Papert in 1969 to show that a single-layer perceptron cannot separate non-linearly-separable data — it took the rediscovery of backpropagation and hidden layers in the 1980s to solve it, which is why XOR remains the smallest possible demonstration that depth (extra hidden layers) genuinely changes what a network can represent, not just how well it fits.
XOR labels a point 1 when exactly one of its two coordinates is positive, which is not linearly separable — no single straight line can divide the four XOR quadrants correctly. This simulation's network uses two ReLU hidden layers precisely because they let it combine several straight-line decision boundaries into a genuinely curved, non-linear boundary, which is the minimum architecture capable of solving XOR at all.
Plain SGD updates each weight by the learning rate times its raw gradient, computed here in the applyOpt function as lr * grad. Momentum instead accumulates an exponential moving average of past gradients (0.9 times the previous momentum plus 0.1 times the new gradient) before scaling by the learning rate, smoothing out noisy updates. Adam goes further, tracking both a first moment estimate m and a second moment estimate v with bias correction, then dividing the learning rate's effect by the square root of v — this adapts the effective step size per-parameter and usually converges fastest of the three.
The right-hand canvas plots training loss in blue and validation loss in red on the same axes, recomputed every 10 epochs from the held-out 20% validation split created in makeDataset. Overfitting is visible the moment the blue training loss keeps falling while the red validation loss flattens out or starts rising — the network is fitting quirks of the training batch rather than the underlying pattern, and increasing L2 regularisation λ or reducing hidden units per layer are the two controls that most directly counteract it.
The L2 regularisation strength λ is added to every weight's gradient in trainStep as lambda * row[j] before the optimiser update, which is mathematically equivalent to shrinking every weight slightly toward zero on each step (weight decay). Larger λ values keep the network's weights small and its decision boundary smoother, which helps on noisy datasets like Moons and Circles but can prevent the network from fitting a genuinely intricate boundary like Spiral if set too high.
The Spiral preset interleaves two classes along intertwined spiral arms that wind around the origin multiple times, which requires a decision boundary with far more curvature and twists than the simple ring (Circles) or crescent (Moons) shapes. With only 16 hidden units per layer the network can struggle to represent every spiral turn; increasing hidden units gives the ReLU layers enough independent linear pieces to approximate the boundary's tight, repeated curves.