API Rate Limiter - 3D

Clients on the left, a token-bucket gateway in the middle, the API servers on the right.

Switch to 2D

Presets

Playback

Traffic

How the clients behave.

Rate limiter

Pick the algorithm and its limits.

Client retries

What clients do after a 429.

Actions

Switch algorithm with one click.

Camera

Quality

Layers

Model checks

The science

Why APIs limit requests

A public API has finite capacity. A rate limiter at the gateway admits requests up to a sustained rate r and answers the rest with HTTP 429 Too Many Requests (often with a Retry-After header). This page runs four classic algorithms on the same request stream so you can see how they differ.

The four algorithms

Fixed window: count requests in each window of length W; admit while count < r W. Simple, but a burst split across a window boundary can pass up to 2 r W in one window-length of time.

Sliding window log: keep the timestamps of admitted requests and admit while fewer than r W lie in the last W seconds. Never more than r W in any window; costs memory.

Token bucket: a bucket of capacity b fills at r tokens/s; each request takes one token. tokens(t) = min(b, tokens(t0) + r (t - t0)). It allows a burst of up to b and then enforces r; any window of length W admits at most b + r W.

Leaky bucket (as a queue): requests join a queue of size b that drains at exactly r per second; if it is full the request is rejected. The output is perfectly smooth (at most r W + 1 per window) at the price of queueing delay level / r.

Facts you can check

Under constant overload a token bucket admits b + r T requests in time T. With a full bucket and a constant offered rate lambda > r the first 429 appears after about b / (lambda - r). If both token refills and requests are Poisson, the token count is a birth-death chain with stationary law pi_n proportional to (r/lambda)^n (n = 0..b), so the admit probability is 1 - pi_0 = 1 - (1-x)/(1-x^(b+1)) with x = r/lambda.

Retries and backoff

A rejected client may retry after min(30 s, base * 2^k); with full jitter the delay is uniform in [0, that]. Retries add to the offered load, so attempts per original request exceed 1. Jitter spreads retries in time so they find tokens as they refill: in the retry-wave presets it completes far more original requests, but it does not reduce the number of attempts.

The three checks

(1) Deterministic 4x overload for 100 s against each algorithm's closed-form admitted count. (2) Time to the first 429 against b/(lambda - r). (3) A 400,000-event simulation of the Poisson birth-death token chain against its closed form.

Simplifications

One client population and one limit key (real gateways limit per API key or IP), instantaneous decisions, no clock skew or distributed counters, HTTP overhead ignored. The birth-death check uses Poisson refills only to have an exact closed form; the live limiter refills continuously and deterministically. This is an educational model, not the documentation of any real API.

What to try
  1. Load "Fixed window: the boundary double burst": the worst-window readout hits 2 x the limit. Switch to "Sliding window log" with the same traffic: it never exceeds the limit.
  2. Token bucket with the big burst allowance (b = 150): bursts pass untouched, then the limit bites. Compare with the leaky bucket, which queues the same burst and releases it at r.
  3. Drag the offered rate below r: no 429s at all. Above r: the admitted rate flattens at r no matter how hard clients push.
  4. Load the two retry-wave presets and compare "Originals served": jitter lets nearly all clients eventually succeed.
  5. Watch graph 4: for the current traffic it compares the worst window of all four algorithms with their theoretical bounds.
Loading Blender model...
Offered, admitted and rejected requests per second
Limiter internal state
Retry amplification
Worst admitted window per algorithm vs bound
Made with 🟠 Blender · Three.js