The Kernel Trick: Lifting Data Into a Separable Dimension

Dataset

Feature Map

Lift into feature space 0%

Kernel coefficient γ 1.00

Decision-plane height 0.00

Classifier Readout

Class A Class B

Feature dim.

Points

Accuracy

Misclassified

Margin (z-units)

–

How it works

A linear classifier can only cut a dataset with a straight line (or flat plane). Concentric circles and XOR-style quadrants are famous examples that no straight line in 2D can separate — the classes are tangled together.

The kernel trick works around this by mapping each point x = (x₁, x₂) into a higher-dimensional feature space φ(x), where the same data often becomes linearly separable. This sim visualizes that map explicitly as a third coordinate:

Concentric circles:  φ(x) = (x₁, x₂, γ·(x₁² + x₂²))
XOR quadrants:        φ(x) = (x₁, x₂, γ·x₁·x₂)

Drag Lift from 0% to 100% to morph the points from the flat, tangled 2D plane up into 3D — the inner disk rises into a paraboloid bowl (or the XOR quadrants tilt into a saddle), and a single flat plane can now slice the two classes apart with zero errors.

In a real kernel method (e.g. an SVM with an RBF kernel), φ(x) is never computed explicitly — often it is even infinite-dimensional. Instead the algorithm only ever needs dot products φ(x)·φ(x′), which a kernel function K(x, x′) computes directly from x and x′ in the original space:

RBF kernel:  K(x, x′) = exp(-γ‖x - x′‖²)
Polynomial:  K(x, x′) = (x·x′ + c)^d

That is the trick: all the benefit of a high-dimensional linear boundary, none of the cost of building the high-dimensional vectors.

γ (gamma) — scales how steeply the feature map lifts points; higher γ makes the bowl/saddle sharper and increases the margin between classes.
Decision-plane height — a horizontal plane in feature space; move it (or hit Auto-fit) to find the height that cleanly separates both classes.
Accuracy / Misclassified — recomputed live from each point's true class vs. which side of the plane it currently sits on, so you can watch separability appear as Lift increases.

Class A Class B Decision plane

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.