K-Nearest Neighbors Regression: Predicting by Local Averaging

Query Point

Query X 0.0

Query Z 0.0

Model

Neighbors k 5

Label noise σ 0.30

Live Readouts

Predicted ŷ

—

True f(x,z)

—

|error|

—

RMSE (grid)

—

How it works

k-NN regression predicts a continuous value instead of a class label. Given a query point x and training pairs (x_i, y_i), find the k closest points N_k(x) by Euclidean distance in feature space, then average their targets:

Uniform:   ŷ(x) = (1/k) · Σ y_i,  i ∈ N_k(x)

Distance-weighted:
  w_i = 1 / (d(x, x_i) + ε)
  ŷ(x) = Σ w_i·y_i / Σ w_i,  i ∈ N_k(x)

Unlike linear regression, k-NN fits no global formula — it is a purely local, non-parametric model: the prediction only ever depends on nearby points.

Query X / Z — move the query point (white marker) across the feature plane; thin lines connect it to its k nearest neighbors.
k — small k (k=1) chases individual noisy points (high variance, jagged fit); large k averages over a wide neighborhood (high bias, over-smoothed, flattens toward the global mean).
Distance-weighted — closer neighbors get more say than distant ones (w_i ∝ 1/distance), usually reducing bias versus a plain uniform average for the same k.
Label noise σ — how much random noise is added to each training point's target when the data is generated; more noise means k=1 fits the noise itself.
RMSE (grid) — root-mean-squared error between the k-NN prediction and the true underlying function, sampled over a 12×12 grid across the whole domain — the single number that shows the bias/variance trade-off as you drag k.

Real-world relevance: k-NN regression underlies collaborative-filtering "similar items" predictions, sensor calibration curves, and any case where nearby examples are the best evidence — with no assumption about the shape of the true relationship.

Training point (height = y) k nearest neighbors Query point → prediction

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.