K-Nearest Neighbors Regression (2D): Predicting by Local Averaging

Query Point

Query X 0.0

Query Z 0.0

Or drag directly on the feature-plane panel below to move the query point.

Model

Neighbors k 5

Label noise σ 0.30

Heatmap Shows

Live Readouts

Predicted ŷ

—

True f(x,z)

—

|error|

—

RMSE (grid)

—

How it works

k-NN regression predicts a continuous value instead of a class label. Given a query point x and training pairs (x_i, y_i), find the k closest points N_k(x) by Euclidean distance in feature space, then average their targets:

Uniform:   ŷ(x) = (1/k) · Σ y_i,  i ∈ N_k(x)

Distance-weighted:
  w_i = 1 / (d(x, x_i) + ε)
  ŷ(x) = Σ w_i·y_i / Σ w_i,  i ∈ N_k(x)

Unlike linear regression, k-NN fits no global formula — it is a purely local, non-parametric model: the prediction only ever depends on nearby points. The feature space here is 2D (X, Z); the target y is drawn as color on the top panel and as height on the cross-section below, exactly the same math as the 3D version with the surface height flattened into a heatmap.

Top panel — a heatmap of the model's prediction (or the true function, or the error, pick with the buttons) across the whole plane; green dots are training points, gold dots are the current k nearest neighbors, thin gold lines connect them to the white query marker. Drag anywhere on it to move the query point.
Bottom panel — a cross-section: fixing Z at the query's current value and sweeping X, it plots the true function (dashed) against the k-NN prediction (solid) so you can watch the fit tighten or flatten as you drag k; training points near that Z slice are shown as faint dots, fading out the further their own Z is from the slice.
k — small k (k=1) chases individual noisy points (high variance, jagged fit); large k averages over a wide neighborhood (high bias, over-smoothed, flattens toward the global mean).
Distance-weighted — closer neighbors get more say than distant ones (w_i ∝ 1/distance), usually reducing bias versus a plain uniform average for the same k.
Label noise σ — how much random noise is added to each training point's target when the data is generated; more noise means k=1 fits the noise itself.
RMSE (grid) — root-mean-squared error between the k-NN prediction and the true underlying function, sampled over a 12×12 grid across the whole domain — the single number that shows the bias/variance trade-off as you drag k.

Real-world relevance: k-NN regression underlies collaborative-filtering "similar items" predictions, sensor calibration curves, and any case where nearby examples are the best evidence — with no assumption about the shape of the true relationship.

Training point k nearest neighbors Query point → prediction

cross-section at Z = 0.0