Instead of grid search or manual tuning, evolutionary algorithms treat each hyperparameter combination (e.g. learning rate and regularization strength) as an individual in a population. The purple surface below is a stylized validation loss landscape: hills are bad configurations, valleys are good ones. Each sphere is a candidate configuration; its height and color encode how well it performs.
Evolutionary strategies like CMA-ES and population-based training (PBT) are used to tune real deep learning systems precisely because loss landscapes over hyperparameters are often non-differentiable and multi-modal — gradient descent can't help you choose a learning rate, but a population can search the space in parallel.
A population of candidate hyperparameter configurations searches a 3D validation-loss landscape generation by generation, using either a genetic algorithm or particle swarm optimization.
The purple-green-red surface encodes validation loss over two hyperparameter axes. Each sphere is a candidate configuration; every generation the population is selected, recombined and mutated (or accelerated toward personal/global bests) so it drifts toward the landscape's minima.
Choose Genetic Algorithm or Particle Swarm, pick a smooth or rugged landscape, and tune population size, mutation/exploration and selection pressure. Step one generation at a time or let it evolve automatically, and watch the best-loss marker track the global minimum.
Population-based training and CMA-ES are used in production ML pipelines precisely because hyperparameter loss surfaces are usually non-differentiable and riddled with local minima — exactly what the rugged landscape here simulates.