Multiclass Classification: One-vs-Rest vs One-vs-One

Decomposition Strategy

Dataset

Classes K 4

Cluster spread 0.75

Highlight ambiguous cells

Live Readouts

Binary classifiers

Train accuracy

—

Ambiguous cells

—

Avg. vote margin

—

How it works

A logistic-regression classifier only knows how to separate two classes: score f(x) = w·x + b, predicted class = sign(f(x)). Real problems usually have K > 2 classes, so the binary classifier has to be reused in one of two standard ways:

One-vs-Rest (OvR)
  train K classifiers: f_k(x) = w_k·x + b_k
  each f_k trained on "class k" vs "everyone else"
  predict:  argmax_k f_k(x)

One-vs-One (OvO)
  train C(K,2) = K(K-1)/2 classifiers, one per class pair (i,j)
  each f_ij trained only on points from i and j
  predict:  every f_ij casts one vote for i or j
            class with the most votes wins

OvR trains fewer classifiers (K), but each one sees a lopsided dataset — one class against all the others combined — which can distort its boundary.
OvO trains far more classifiers as K grows (quadratically), but each sees a small, cleanly separable two-class problem and votes are usually decisive.
The vote margin readout is the gap between the winning class's score (OvR) or vote count (OvO) and the runner-up — a small margin means the grid cell sits right on a decision boundary, which is exactly where a real classifier is most likely to misfire on new data.
Every classifier here is trained live with real batch gradient descent on the logistic loss — this is not a lookup table, dragging the spread slider or resampling the dataset genuinely retrains all classifiers.

Real-world relevance: scikit-learn's OneVsRestClassifier and OneVsOneClassifier wrap exactly this logic around any binary classifier (logistic regression, linear SVM, …) to make it handle more than two classes.

Decision surface (height = confidence) Training points (colored by true class)

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.