The heatmap is a hidden 2D hyperparameter space — learning rate on the horizontal axis, regularization strength on the vertical axis — colored by the Gaussian process (GP) surrogate's posterior mean validation accuracy, fit exactly (RBF kernel, Gauss–Jordan matrix inversion) to every trial run so far. Cells fade toward grey where the surrogate is uncertain — far from any sampled point.
- Dots — completed trials (the newest one highlighted); each real evaluation is expensive, so Bayesian optimization tries to use as few as possible.
- Pulsing marker — the point the acquisition function (Upper Confidence Bound: mean + κ·σ) currently rates highest.
- κ (kappa) — a higher value rewards uncertainty more, pushing exploration into unsampled territory; a lower value chases the best mean found so far (pure exploitation).
- Lengthscale — how far a single trial's information is assumed to generalize; shorter values produce a bumpier, more localized fit.
- Show true landscape — reveals the hidden ground-truth surface (never visible to a real optimizer), for comparison against the surrogate.