Grid search is the most literal way to tune a model: pick a discrete set of values for each hyperparameter, form every possible combination, and evaluate all of them. Here the floor is a lattice over two hyperparameters — learning rate (x-axis, log scale) and regularization strength (z-axis, log scale). Every lattice intersection is one full training-and-validation run. A scan head sweeps the grid in the same nested-loop order the algorithm actually uses — every value of regularization for the first learning rate, then every value for the second, and so on — growing a bar whose height and color encode the validation score found there.
resolution² — a small bump in resolution multiplies the work.k times, so total model fits = combinations × folds.GridSearchCV tracking best_score_ as it works through param_grid.
Because cost scales as (grid points)d for d hyperparameters,
exhaustive grid search becomes impractical past 3–4 dimensions — the "curse of
dimensionality" is exactly why random search, Bayesian optimization and successive
halving exist as cheaper alternatives once the search space grows.
A 3D lattice of every learning-rate/regularization combination gets swept in strict nested-loop order, growing colored bars for each validation score and tracking the best combination found so far.
Grid search tries every point on a fixed lattice exhaustively — cost grows as resolution² per axis and multiplies again by cross-validation folds, while a random-search overlay shows how the same budget looks scattered instead of gridded.
Raise the grid resolution or fold count and watch total model fits explode, adjust scan speed, and toggle the random-search overlay to compare coverage patterns for the same evaluation budget.
Because grid search cost scales as (grid points)^d, going from 2 to just 4 hyperparameters at 10 values each turns 100 model fits into 10,000 — the "curse of dimensionality" that motivates random and Bayesian search.