The Universal Approximation Theorem says a feed-forward network with a single hidden layer of sigmoid neurons can approximate any continuous function on a bounded domain arbitrarily closely, given enough neurons. Each hidden neuron computes a shifted, scaled S-curve, and the output layer is just a weighted sum:
neuron i: h_i(x) = σ(s_i·x + t_i), σ(z) = 1/(1+e^-z)
output: f(x) = c + Σ w_i · h_i(x)
loss: MSE = (1/M) Σ (f(x_m) − target(x_m))²
Every frame this page runs one batch-gradient-descent step over M sample points, nudging every s_i, t_i, w_i and c downhill on the MSE surface via the chain rule (σ′(z) = σ(z)(1−σ(z))). This 2D version lays the same computation out as four linked panels instead of one 3D scene: the top panel is the function fit itself, the ridgeline unstacks every neuron's own contribution as a joyplot (the 2D analogue of stacking curves back in a third axis), the loss panel tracks MSE across training steps so you can see convergence (or its absence) directly, and the weight-space panel plots each neuron's own (s, t) — its slope and shift — with dot size showing |w|, so you can watch neurons move through parameter space as they specialize.
- Neurons N — fewer neurons cap how sharp a curve the network can represent; watch the Step or Sawtooth targets fail to fit cleanly below ~4 neurons, then clean up as N grows.
- Learning rate η — too high and the fit oscillates or diverges (watch the loss panel spike); too low and it converges too slowly to see.
- Batch samples M — how many x-points the gradient is averaged over each step; very few samples makes the loss curve noisier and the fit less faithful between sample points.
- Weight-space panel — pan and zoom it freely; neurons that end up far apart in (s, t) space have specialized to different regions of the input domain.