Hamiltonian Monte Carlo: Leapfrog Sampling

Target Posterior

Sampler Parameters

Leapfrog step size ε 0.12

Leapfrog steps L 18

Chain Diagnostics

Samples drawn

Accept rate

—

Hamiltonian H = U + K (proposal)

—

θ₁

0.00

θ₂

0.00

How it works

Bayesian inference needs samples from a posterior p(θ) that usually has no closed form. Hamiltonian Monte Carlo (HMC) draws those samples by borrowing physics: it treats −log p(θ) as a potential-energy surface U(θ) and lets a fictitious particle roll across it.

Hamiltonian:  H(θ,r) = U(θ) + K(r),   K(r) = ½‖r‖²
Step:  1. sample momentum   r ~ N(0, I)
       2. leapfrog integrate (θ,r) for L steps of size ε:
            r ← r − (ε/2)∇U(θ)
            θ ← θ + ε r
            r ← r − (ε/2)∇U(θ)
       3. accept θ* with prob. min(1, exp(H(θ,r) − H(θ*,r*)))

Because leapfrog conserves H almost exactly, long, high-momentum trajectories still land with a high acceptance probability — letting HMC glide down narrow, correlated ridges that make plain random-walk Metropolis crawl step by step. The 3D surface below is the density p(θ) itself (peaks = high posterior probability); the sampler physically integrates on U = −log p, so it rolls into U's valleys, which are exactly the density's peaks.

ε (step size) — too large and leapfrog's discretisation error blows up H, crashing the accept rate; too small wastes computation taking tiny steps.
L (leapfrog steps) — how far the particle travels per proposal; longer trajectories explore more per sample but cost more gradient evaluations.
Target — the correlated Gaussian is the textbook case where HMC's use of the gradient beats naive random-walk proposals; the bimodal mixture shows where even HMC struggles to jump between separated modes.

This is the algorithm (and its descendant, the No-U-Turn Sampler) underneath Stan, PyMC and NumPyro — the standard engines for fitting real Bayesian models today.

Chain state / trajectory Accepted sample Rejected proposal

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.