Training a model well often depends less on the architecture than on the hyperparameters around it — learning rate, regularization strength, tree depth, and so on. Searching that space efficiently is itself an optimization problem. This simulation renders a synthetic 2-D hyperparameter space as a 3D loss landscape (lower = better) and shows how four common search strategies explore it under the same trial budget.
Bergstra & Bengio's 2012 paper "Random Search for Hyper-Parameter Optimization" showed that random search often finds equally good hyperparameters as grid search using only a fraction of the trials, simply because most hyperparameters barely affect performance — so grid search's exhaustiveness is largely wasted effort.
A 3D loss landscape shows how grid search, random search, Bayesian optimization and genetic algorithms explore a hyperparameter space differently under the same trial budget.
Each strategy places trial markers on the loss surface using its real placement logic — fixed lattice, uniform random, exploration/exploitation around the current best, or an evolving population — so you can watch how quickly each converges on the global minimum.
Pick an algorithm, set the trial budget and landscape ruggedness, then watch trials populate the surface. The gold beacon marks the best loss found so far; the stats panel tracks how many trials it took to find it.
Bergstra & Bengio (2012) showed random search often matches grid search's results with far fewer trials, because most hyperparameters have little effect on performance — grid search wastes budget exploring them exhaustively.