Differential Evolution: an Optimizer That Needs No Derivatives

Optimizing without a gradient

Gradient descent needs a differentiable objective function and, ideally, a well-behaved one. Many real-world objectives fail that test outright — they come from a black-box simulation, contain discrete decisions, or are simply too messy to differentiate. Differential Evolution (DE), introduced by Rainer Storn and Kenneth Price in 1997, sidesteps the whole issue: it evaluates the objective function only at points, never its derivative, and moves a whole population of candidate solutions using nothing more than arithmetic on the population itself.

The DE/rand/1/bin variant

The classic and most widely used variant, named DE/rand/1/bin for its mutation and crossover style, does three things to every member of the population each generation:

for each target vector x_i in the population:
  pick r1, r2, r3 distinct, all != i, at random from the population

  mutant  v = x_r1 + F * (x_r2 - x_r3)          // F: differential weight

  trial u_j = v_j  if rand() < CR (or j == random index)
            = x_i_j  otherwise                   // CR: crossover rate

  if f(u) is better than f(x_i):  x_i (next gen) = u
  else:                            x_i (next gen) = x_i   // selection

The mutation step is the namesake move: a difference between two randomly chosen population members, scaled by F, becomes a perturbation added to a third. Crossover then mixes the mutant with the original target vector, one dimension at a time, controlled by CR. Selection is greedy — the trial replaces the target only if it strictly improves the objective, so the population's best-known fitness never gets worse from generation to generation.

Why the difference vector is the right scale

The elegance of DE's mutation is that the step size is self-adapting without any extra machinery: early in the search, when the population is spread widely across the search space, the differences x_r2 − x_r3 are large, and mutation takes big exploratory steps. As the population converges toward a promising region, those same differences shrink automatically, and mutation naturally refines the search with smaller steps. No separate cooling schedule, as in simulated annealing, is needed — the population's own diversity supplies it.

Self-adaptive variants

Classic DE still leaves F and CR as fixed hyperparameters that must be tuned per problem. Later variants remove even that: SaDE (self-adaptive DE) and JADE encode F and CR as evolvable traits attached to each individual, sampled from a distribution that itself adapts based on which values of F and CR produced successful trial vectors in recent generations. This closes the loop entirely — the algorithm learns its own step-size and mixing-rate schedule from the shape of the problem it is currently solving, at the cost of a small amount of extra bookkeeping per generation.

Benchmarking on standard test functions

DE is routinely evaluated on a small set of adversarial test functions. Rosenbrock's function has a narrow, curved valley leading to its minimum — easy to find the valley, slow to crawl along it. Rastrigin's function is riddled with regularly spaced local minima on top of a smooth bowl, testing whether an optimizer can avoid getting trapped. Ackley's function is nearly flat far from the origin with a sharp central well, testing whether an optimizer can find the signal in a large, almost featureless plateau. DE reliably finds the global minimum of all three given enough generations, though its convergence on Rosenbrock in particular is markedly slower than a gradient-based method that can exploit the valley's local slope directly — a fair trade for not needing that slope to exist or be computable at all.

Frequently asked questions

Try it live

Everything above runs in your browser — open Differential Evolution Optimizer and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.