The bumpy surface is a synthetic hyperparameter loss landscape L(x, z). AutoML systems search this space for the configuration that minimizes validation loss, without a human hand-tuning each run.
L(x,z) = bowl(x,z) + complexity · Σ Aᵢ·sin(fᵢx)·cos(fᵢz)
Random: sample θ ~ Uniform(domain)
Grid: sweep θ over a fixed lattice
Smart: θᵢ ← θᵢ + step·(best − θᵢ) + noise (PSO-style exploitation)
- Search strategy — Random samples blindly; Grid sweeps systematically; Smart (PSO) pulls agents toward the current best point like a simplified Bayesian/evolutionary optimizer.
- Search agents — number of hyperparameter configurations evaluated in parallel each round.
- Step size — how far each agent moves per iteration (exploration vs. fine-tuning trade-off).
- Landscape complexity — adds more local minima/ridges, mimicking a harder, less smooth hyperparameter space.
- Restart Search — resets all agents and best-loss tracking.
Real application: AutoML frameworks (e.g. Bayesian optimization, evolutionary search, or random/grid search baselines) automate exactly this kind of search over learning rate, model depth, regularization, etc.