The floor grid is a 2D hyperparameter space (learning rate on one axis, regularization strength on the other) and height is validation accuracy. In real tuning you never see this landscape directly — each evaluation means fully training a model, so it is expensive. Bayesian optimization fits a Gaussian process (GP) surrogate to the trials run so far, then uses an acquisition function to decide where to try next, balancing high predicted accuracy against high uncertainty.
Bayesian optimization typically needs only a few dozen trials to approach the best setting in a landscape where grid or random search would need thousands of expensive full training runs — which is why it underlies tools like Optuna, Hyperopt and Google Vizier.
A Gaussian-process surrogate model searches a hidden hyperparameter landscape trial by trial, using an acquisition function to trade off exploring uncertain regions against exploiting the best result found so far.
The colored surface is the surrogate's posterior mean; the pale wireframe shell above it is its uncertainty envelope. Together they show how Bayesian optimization models both "what looks good" and "how sure are we" before spending another expensive trial.
Raise exploration (κ) to send trials into uncertain territory, or lower it to chase the current best. Adjust the GP lengthscale to see how smoothness assumptions reshape the fit, then click "Sample suggested point" repeatedly to watch the surrogate converge on the true optimum.
Bayesian optimization often needs only a few dozen trials to approach the best hyperparameter setting in a space where grid search would require thousands of full training runs.