Hidden performance surface & each method's trials

Grid search

Random search

Bayesian optimisation

Best-so-far vs. trial number