Every point (x₁,x₂) is fed through one hidden layer and a sigmoid output neuron:
aₕ = f(W₁·x + b₁)
ŷ = σ(W₂·aₕ + b₂)
where f is the selected activation (sigmoid, tanh or ReLU) and σ is always sigmoid so the output reads as a class probability. Every frame runs one step of full-batch gradient descent on binary cross-entropy loss:
Δw = η · ∂Loss/∂w
The gradient ∂Loss/∂w is found by backpropagation — the chain rule applied layer by layer, back-to-front, which is exactly the Δw formula from the article. Watch the decision boundary bend to fit the data as the loss curve/accuracy improve — that bending is learning.
- Weights: connection strengths, updated every step.
- Bias: per-neuron offset, lets a neuron fire even when inputs are zero.
- A higher η learns faster but can overshoot and destabilize; ReLU trains faster than sigmoid/tanh but can "die" (stuck at zero) on some units.