🧠 Supervised Learning Algorithms

About this simulation

Every supervised classifier does the same fundamental job: it looks at labelled training points scattered across a feature space and builds a rule for guessing the label of any new point. This scene renders that rule as a literal 3D landscape — a probability surface hovering over a 2D feature plane. Hills mean "confidently class B," valleys mean "confidently class A," and the blurry slope between them is the decision boundary.

Training points are drawn as small spheres coloured by their true label and lifted to their true class height. When a sphere sits over a valley but is coloured for the "hill" class (or vice-versa), that point is being misclassified — you can see the model's mistakes directly instead of reading them off a confusion matrix.

How it works

• Logistic Regression — fits a smooth linear (or low-order polynomial) boundary by gradient descent. Complexity controls how many polynomial feature terms it is allowed to use, so higher values let the boundary curve.
• k-Nearest Neighbors — classifies each point of the landscape by a majority vote of its k closest training points. Complexity here is inverted: a high complexity value means a small k, which memorises local noise and produces a jagged, overfit surface.
• Decision Tree — recursively splits the feature plane along axis-aligned cuts that best separate the classes (a simplified Gini-impurity CART). Complexity sets the maximum tree depth, producing the tree's characteristic staircase boundaries.
• Random Forest — trains several shallow trees on bootstrap-resampled subsets of the data and averages their votes, smoothing out any single tree's overfitting.

Controls

• Algorithm — switch between the four classic supervised learners.
• Dataset shape — moons and circles need a non-linear boundary; blobs are linearly separable.
• Model complexity — pushes each algorithm from underfitting (too simple) toward overfitting (memorising noise).
• Label noise — randomly flips a percentage of training labels, so you can see which algorithms are robust and which chase the noise.
• Training points — how many labelled examples the model gets to learn from.
• ↺ Resample data — draws a fresh random dataset with the same settings.

Did you know?

There is no single "best" algorithm for every dataset — the "no free lunch" theorem shows that averaged across all possible problems, every algorithm performs the same. What matters in practice is matching an algorithm's bias (its assumptions about the shape of the boundary) to the actual structure of your data.