Universal Approximation Theorem: Neurons Fitting a Curve

Hidden Layer

Neurons N 8

Learning rate η 0.40

Target function f(x)

Live Fit

MSE loss

—

Steps

Active neurons

Worst point err

—

How it works

The Universal Approximation Theorem says a feed-forward network with a single hidden layer of sigmoid (or ReLU) neurons can approximate any continuous function on a bounded domain arbitrarily closely, given enough neurons. Each hidden neuron computes a shifted, scaled S-curve, and the output layer is just a weighted sum:

neuron i:   h_i(x) = σ(s_i·x + t_i),  σ(z) = 1/(1+e^-z)
output:     f(x) = c + Σ w_i · h_i(x)
loss:       MSE = (1/M) Σ (f(x_m) − target(x_m))²

Every frame this page runs one batch-gradient-descent step over M sample points, nudging every s_i, t_i, w_i and c downhill on the MSE surface via the chain rule (σ′(z) = σ(z)(1−σ(z))). You are watching backpropagation happen in real time on a network small enough to see completely.

Faint colored curves — each individual neuron's own output h_i(x)·w_i, stacked back in Z so you can see them separately.
Bright teal curve — the network's actual output f(x): the sum of every neuron.
White dashed curve — the target function you're asking the network to learn.
Colored dots — sampled points on f(x), tinted from green (near-zero error) to red (large error) at that x.
Neurons N — fewer neurons cap how sharp a curve the network can represent; watch the Step or Sawtooth targets fail to fit cleanly below ~4 neurons, then clean up as N grows.

Network output f(x) Target function Individual neurons

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.