How it works ▾
The ghost wireframe is a hidden true objective f(λ) over two hyperparameters
(e.g. log learning-rate, log regularization). The n gold spheres are noisy
observations of it. The solid colored surface is a kernel-weighted surrogate:
ŝ(λ) = Σᵢ wᵢ(λ)yᵢ / Σᵢ wᵢ(λ), wᵢ(λ) = exp(−‖λ−λᵢ‖² / 2ℓ²)
Surface color encodes pointwise error ε(λ) = |f(λ) − ŝ(λ)| (green = accurate, red = poor).
Lengthscale ℓ trades bias for variance: too small and ŝ chases noise between
sparse points (high variance); too large and it oversmooths real structure (high bias) —
MSE = Bias² + Variance. More observations shrink both, and for
smooth f the uniform error falls roughly as
‖f − ŝ‖∞ = O(n^−α). Noise σ corrupts each observed value,
the direct analogue of a stochastic/expensive objective in real hyperparameter search.