AutoML must solve the CASH problem (Combined Algorithm Selection and Hyperparameter optimization): choose both a model family and its hyperparameters at once. Fully training every candidate to convergence is too expensive, so real AutoML systems (auto-sklearn, Hyperband, ASHA) use Successive Halving: give every candidate a small training budget, keep only the top performers, then give the survivors a bigger budget.
n_k = floor( n_0 / η^k ) candidates alive after round k
b_k = b_0 · η^k training budget per candidate at round k
score_est(i, b) ≈ q_i + N(0, σ/√b) noisy validation estimate
Each colored bar is one candidate pipeline (an algorithm family + one hyperparameter draw), with a true underlying quality qi hidden from the search. At every round the search only sees a noisy estimate of qi — the more compute budget b a candidate gets, the tighter that estimate becomes (variance shrinks as 1/b). Bars are ranked by their current estimate, the bottom 1 − 1/η fraction is eliminated and sinks away, and the remaining pipelines move on to a round with η× more budget each — so total compute stays roughly constant per round while resolution on the survivors increases.
- η (halving factor) — how aggressively weak pipelines are pruned each round; higher η eliminates faster but risks cutting a candidate that only looked bad due to noise.
- n₀ — the size of the initial portfolio drawn across algorithm families.
- σ — how noisy a single low-budget evaluation is; higher σ makes early rounds less reliable, which is exactly why more budget later is needed to confirm the winner.
This is the same idea behind Hyperband (Li et al., 2018) and ASHA — it lets AutoML explore many algorithm families cheaply and spend expensive full-budget training only on the few pipelines that already look promising. This 2D view renders the same leaderboard race as a bar chart you can pan and zoom into.