Comparing Hyperparameter Optimization Algorithms (2D)

This 2D companion runs the exact same four search strategies as the 3D loss-landscape version, but reads them from directly above instead of orbiting a mesh: a heatmap (indigo = low loss, red = high loss) stands in for the 3D surface, trial points land on it as dots, the current best is a pulsing gold ring, and the most recent trial glows red — making it easier to trace exactly which region each algorithm is probing next. Grid search marches a fixed lattice, random search scatters uniformly, the Bayesian-style search narrows a shrinking radius around the best point found so far, and the genetic algorithm visibly clusters its population toward low-loss basins generation by generation. Reseeding the landscape or raising its ruggedness changes which strategy wins the race to a good minimum.