The terrain is a synthetic loss surface with one deep global minimum and a few shallow decoys, plus a wavy ruggedness term you control: L(x,z) = λ(x²+z²) − Σᵢ Aᵢ·exp(−‖(x,z)−cᵢ‖²/2σᵢ²) + r·noise(x,z) Valleys (green) are good hyperparameter combinations, peaks (red) are bad ones. Four simplified search strategies race across it:

• Grid — sweeps a fixed lattice of points left-to-right, systematic but blind to where the landscape actually dips.
• Random — every point is an independent uniform draw; no memory of past results.
• Bayesian — candidates cluster near the best point found so far, with search radius σ shrinking as iterations pass and a small chance of a fully random probe — the same explore/exploit trade-off a Gaussian-process acquisition function makes, without the full posterior model.
• CMA-ES — a population sampled from a Gaussian around a running mean; the better half becomes the new mean, and the step size σ grows after an improving generation and shrinks otherwise: xₜ ~ m + σ·N(0,I), m ← mean(elites), σ ← σ·(improved ? 1.08 : 0.92)

Real pipelines add population-based training (agents periodically copy and perturb the best performer's hyperparameters mid-training) and use the Neural Tangent Kernel to reason about how a network's training dynamics change as hyperparameters do — both build on the same explore/exploit idea shown here.