The Perceptron: Learning a Line, One Mistake at a Time

A neuron that only says yes or no

Frank Rosenblatt's perceptron (1958) is the ancestor of the modern artificial neuron: it takes an input vector x, computes a weighted sum plus a bias, and passes the result through a hard threshold. If the sum is positive it fires (output +1); otherwise it does not (output -1 or 0, depending on convention). Geometrically, the weight vector w and bias b define a hyperplane that slices the input space in two, and the perceptron simply reports which side a point lands on.

predict(x) = sign( w·x + b )     // +1 if w·x + b > 0, else −1

decision boundary:  w·x + b = 0    (a line in 2D, a hyperplane in general)

The learning rule: only wrong answers move the line

Training is deliberately minimal. Present one labelled example (x, y) at a time, with y = +1 or -1. If the current weights already classify it correctly, do nothing. If they get it wrong, nudge the weights in the direction that would have made that example more correct:

for each training example (x, y):
  if sign(w·x + b) != y:            // misclassified
    w ← w + eta * y * x              // eta = learning rate
    b ← b + eta * y
// correctly classified points cause no update at all

Each update rotates the decision boundary a little toward the misclassified point's correct side, without ever touching examples the model already gets right. Sweep repeatedly over the training set (each full sweep is an "epoch") and, under the right conditions, the line eventually settles where every point is correctly classified.

The convergence theorem

The perceptron convergence theorem, proved not long after Rosenblatt introduced the algorithm, guarantees that if the training data is linearly separable - if some hyperplane really does put every positive example on one side and every negative example on the other - then this simple mistake-driven procedure finds such a hyperplane in a finite number of updates, and the bound on that number depends only on the margin between the two classes and the size of the data, not on how many features there are. It was, in 1958, a genuinely striking result: a trivially cheap update rule with a hard convergence guarantee.

The wall: XOR, and Minsky and Papert

The guarantee has a matching failure mode. If no hyperplane separates the classes, the weights never converge - they keep oscillating, correcting one mistake at the cost of creating another, forever. The textbook example is the XOR function: its four points (0,0)→0, (0,1)→1, (1,0)→1, (1,1)→0 have the two positive points on opposite corners of the unit square, so no single straight line can isolate them from the two negative points. Marvin Minsky and Seymour Papert's 1969 book Perceptrons proved this limitation rigorously and is widely credited with cooling funding and interest in neural networks for over a decade - a period often called the first AI winter.

The way out: stack more of them

The fix Minsky and Papert already understood in principle is to chain perceptron-like units into layers: a hidden layer can carve the input space into pieces that a single output unit then recombines, and two hidden units are already enough to solve XOR by drawing two lines instead of one. What was missing until the 1980s was a practical way to train the hidden layers, since the simple mistake-driven rule has nothing to say about units that are not directly connected to the labelled output. Replacing the hard threshold with a smooth activation function and training by gradient descent through the chain rule - backpropagation - is the idea that turned Rosenblatt's single neuron into the deep networks used today.

Frequently asked questions

Try it live

Everything above runs in your browser — open The Perceptron and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.