Every model has knobs a training run cannot learn on its own — learning rate, regularization strength, tree depth, dropout. The surface shown here is a stand-in validation-loss landscape over two such hyperparameters: warmer, higher terrain means worse validation loss, and the deep blue basins are good configurations. A real tuning workflow never sees this surface directly — it can only query points on it, one training run at a time, which is exactly what the markers below are doing.
On rugged, non-convex validation surfaces — common with deep networks — Bayesian optimization typically needs far fewer trials than grid search to reach a comparable loss, which matters a lot when a single "trial" can mean hours of GPU training.
A 3D validation-loss landscape over two hyperparameters, searched live by grid search, random search or a simplified Bayesian optimizer — watch each strategy hunt for the minimum under the same trial budget.
The colored terrain encodes validation loss across a learning-rate/regularization grid: blue basins are good configurations, red ridges are bad ones. Each marker is one training run's result; the gold marker tracks the best trial found so far.
Pick a search strategy, set the trial budget and landscape ruggedness, then watch trials populate the surface. Toggle the search path to see the order trials were evaluated in, and compare how fast each strategy converges.
Grid search scales exponentially with the number of tuned hyperparameters, which is why most production ML pipelines default to random search or Bayesian/sequential model-based optimization once past two or three dimensions.