CMA-ES (Covariance Matrix Adaptation Evolution Strategy) is a derivative-free optimizer for black-box functions. Each generation it draws λ candidates from a multivariate Gaussian, evaluates them, and reshapes the Gaussian toward the better ones:
x_i = m + σ · B D z_i, z_i ~ N(0, I)
m' = Σ w_i x_i:μ (weighted mean of the μ best)
p_σ' = (1-c_σ)p_σ + √(c_σ(2-c_σ)μeff) · C^(-1/2) (m'-m)/σ
σ' = σ · exp( (c_σ/d_σ)(‖p_σ‖/E‖N(0,I)‖ - 1) )
p_c' = (1-c_c)p_c + h_σ√(c_c(2-c_c)μeff) · (m'-m)/σ
C' = (1-c1-cμ)C + c1·p_c p_c^T + cμ·Σ w_i y_i y_i^T
- Mean update — the distribution's center moves to the weighted average of the μ = λ/2 fittest samples, so search drifts toward better regions.
- Covariance adaptation (C) — the rank-μ term stretches the sampling ellipse along directions that produced good steps, letting the search self-align with curved valleys (this is why it tracks Rosenbrock's banana-shaped valley far better than isotropic random search). Turn the toggle off to freeze C = I and compare.
- Step-size control (σ, cumulative path p_σ) — compares the length of the accumulated search path to its expected length under random selection: consecutive steps pointing the same way grow σ, back-and-forth steps shrink it. Turn the toggle off to fix σ at σ₀ and compare.
- The purple ellipse on the contour map is the 1-σ contour of the current sampling Gaussian, from the eigendecomposition C = B D² Bᵀ. The chart below plots best fitness and σ on a log scale, generation by generation.
Real-world relevance: this is the algorithm behind many production hyperparameter-tuning and neural-architecture-search backends when gradients aren't available — robotics gait tuning, reinforcement-learning policy search, and black-box engineering design all use the same update rules shown here.
Drag the contour map to pan, scroll/pinch to zoom — it's an independent 2D re-derivation of the 3D landscape view, not a flattened screenshot of it.