A counterintuitive, well-established result
A widely cited 2012 finding by Bergstra and Bengio showed that random search often outperforms grid search for hyperparameter tuning — surprising, given that grid search feels more systematic and thorough. The explanation comes down to a common real-world fact: usually only a few hyperparameters actually matter much for a given problem, while several others have comparatively little effect.
Where grid search wastes its budget
When only one hyperparameter (say, learning rate) meaningfully affects performance and another (say, a rarely-touched regularization detail) doesn't, grid search still spends its trial budget evenly across both, effectively repeating many near-identical trials on the axis that matters while thoroughly covering the axis that doesn't. Random search, sampling every hyperparameter independently and continuously, almost never repeats a value on the important axis, so a far larger share of its budget contributes genuinely new information.
When grid search is still fine
If every hyperparameter genuinely matters roughly equally, grid search's even, exhaustive coverage can be perfectly competitive with random sampling. The Bergstra and Bengio result specifically applies when relevance is concentrated in a small number of dimensions — common in practice, but not a law of nature for every problem.
What beats both
Bayesian optimization uses the results of previous trials to intelligently choose where to sample next, rather than sampling blind on either a grid or at random, generally finding good hyperparameters with fewer total trials than either method. It's the more sophisticated approach explored alongside grid and random search in this site's AutoML Lab.
🧪 Try it yourself: the Hyperparameter Tuning Lab simulation lets you experiment with everything described above directly in your browser.