Nelder–Mead Simplex Optimization

Loss Surface

Simplex Search

Steps per second 4

State

Last move: —

Iteration

Simplex size

—

Best f(x, y)

—

How it works

Nelder–Mead is a derivative-free direct-search method for minimizing a function of several variables. In 2D it keeps a moving triangle ("simplex") of 3 trial points, ranked best (lowest f), good (middle) and worst (highest). Every iteration replaces the worst point using only function evaluations — no gradient is ever computed:

c  = centroid of all points except worst
xr = c + α(c − worst)              reflect   (α = 1)
xe = c + γ(xr − c)                 expand    (γ = 2, tried if xr beats best)
xc = c + ρ(xr − c) or ρ(worst − c) contract  (ρ = 0.5)
shrink: every point → best + σ(point − best)   (σ = 0.5, last resort)

Reflect — if the mirrored point is merely better than "good", accept it and move on.
Expand — if reflecting found new territory better than the current best, push further that direction.
Contract — if reflecting didn't help, shrink the step toward the centroid instead.
Shrink — if even contracting fails, collapse the whole triangle toward the best point.

Real-world relevance: this is the classic fminsearch-style optimizer used when a cost function is expensive, noisy or non-differentiable — common in chemical process tuning, curve-fitting instrument calibrations and control-loop tuning, where you can evaluate f(x) but can't (or don't want to) differentiate it. It converges fast on smooth bowls, but — as the Rastrigin surface shows — it can stall in a local dip on a landscape full of ripples.

Best point Good point Worst point

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.