Hamiltonian Monte Carlo: Phase-Space Leapfrog (2D)

Target Posterior

Sampler Parameters

Leapfrog step size ε 0.12

Leapfrog steps L 18

Chain Diagnostics

Samples drawn

Accept rate

—

Hamiltonian H = U + K (proposal)

—

θ₁

0.00

θ₂

0.00

Max |ΔH| this trajectory (energy drift)

—

How it works

This 2D view splits Hamiltonian Monte Carlo into the two things a physicist actually looks at: the left panel is the posterior itself, a contour map of p(θ) with the chain's accepted path traced across it; the right panel is phase space — the (θ₁, momentum r₁) plane where HMC's real dynamics live, plus a live energy trace beneath it.

Hamiltonian:  H(θ,r) = U(θ) + K(r),   K(r) = ½‖r‖²
Step:  1. sample momentum   r ~ N(0, I)
       2. leapfrog integrate (θ,r) for L steps of size ε:
            r ← r − (ε/2)∇U(θ)
            θ ← θ + ε r
            r ← r − (ε/2)∇U(θ)
       3. accept θ* with prob. min(1, exp(H(θ,r) − H(θ*,r*)))

Because leapfrog is symplectic, its energy error per step is O(ε²) and, critically, does not drift secularly over long trajectories — it oscillates around the true value. The energy-trace panel plots H at every leapfrog half-step of the current proposal so you can watch that near-conservation directly: a smooth, bounded wiggle means a healthy acceptance probability is coming; a trace that visibly climbs means ε is too large and the proposal will likely be rejected. The phase-space orbit shows why HMC beats a random walk on a correlated target — for the correlated Gaussian it traces near-elliptical arcs that sweep quickly along the ridge instead of diffusing step by step.

ε (step size) — too large and leapfrog's discretisation error blows up H, crashing the accept rate (watch the energy trace spike); too small wastes computation taking tiny steps.
L (leapfrog steps) — how far the particle travels per proposal in phase space; longer trajectories explore more per sample but cost more gradient evaluations.
Target — the correlated Gaussian is the textbook case where HMC's use of the gradient beats naive random-walk proposals; the bimodal mixture shows where even HMC struggles to jump between separated modes.

This is the algorithm (and its descendant, the No-U-Turn Sampler) underneath Stan, PyMC and NumPyro — the standard engines for fitting real Bayesian models today.

θ-space: posterior density & chainPhase space (θ₁, r₁) + energy trace

Live leapfrog trajectory Accepted sample Rejected proposal Energy trace H(t)