Neural Network Decision Lab

Train a real small network, watch its neurons and decision boundary, then turn its output into a noisy drift-diffusion choice.

Presets

Playback

Training

Input features

Data

Decision model (drift-diffusion)

Layers

Model check

The science

The network

A multilayer perceptron maps the chosen input features (x, y, x², y², xy, sinπx, sinπy) through hidden layers a(l) = f(W(l)a(l−1) + b(l)) to one output logit z; the probability of class 1 is p = 1/(1 + e−z). Training minimises the cross-entropy L = −mean[y ln p + (1−y) ln(1−p)] + (λ/2)ΣW² by back-propagation (the chain rule, output error δ = p − y) with plain SGD, momentum or Adam on random mini-batches. 25 % of the points are held out as a test set; the gap between train and test loss shows over-fitting.

Reading the picture

The colour map is p over the whole plane: the decision boundary is where it turns from blue to orange. Each hidden neuron has its own little map of what it responds to; deeper neurons combine the simpler shapes of earlier ones. Line width is |weight|. Drag the probe to feed a new stimulus and watch every activation, number and pulse change. Why hidden layers matter: XOR cannot be separated by a single linear boundary, but a hidden layer (or the feature xy) solves it.

From network output to a decision (drift-diffusion)

Brains do not decide instantly: noisy evidence is accumulated until a threshold is hit. In the drift-diffusion model dx = v dt + σ dW the accumulator x starts at 0 and the choice is made when it reaches ±a. Here the drift is the network's confidence, v = k · z. For this process the probability of hitting the upper bound is P = 1/(1 + e−2va/σ²) and the mean decision time is E[T] = (a/v) tanh(av/σ²); the page adds a fixed non-decision time t₀ for sensory and motor delays. A higher threshold makes choices slower but more accurate (speed-accuracy trade-off); a probe near the boundary has v≈0 and gives slow, uncertain decisions, as in real reaction-time data.

Independent checks (shown in the page)

(1) The back-propagated gradient of every sampled weight and bias equals a central finite-difference derivative of the loss (relative error far below 10⁻⁴). (2) A Monte Carlo simulation of 5000 diffusion trials reproduces the closed-form choice probability within four binomial standard errors. (3) The simulated mean decision time matches (a/v) tanh(av/σ²) within 5 % (the simulation uses a continuity-corrected barrier to remove the Euler step bias).

Honest simplifications

This is a tiny two-dimensional toy: real neurons spike, have dendrites and learn with local, not global, rules, and back-propagation is not how cortex learns. Linking the artificial logit to a drift rate is a modelling choice (a hypothesis in cognitive neuroscience, not a measured law), the gain k, threshold a, noise σ and t₀ are free illustrative parameters, and the accumulator has no leak, bias or collapsing bounds. Datasets are synthetic; accuracy numbers say nothing about real tasks.

Loss: train vs test
Accuracy: train vs test
Probability of choosing class 1 along the probe row
Decision-time histogram at the probe