🚦 API Rate Limiter: Token Bucket, Fixed & Sliding Window

It is the cheapest to implement and reason about, requiring only a single integer counter and a reset timestamp per client, with negligible storage overhead even at massive scale. For rate limits that exist mainly as a coarse abuse-prevention backstop rather than a precise SLA guarantee, the boundary burst weakness is often an acceptable trade-off against its simplicity and low cost.

No, they are near-opposites in effect despite sharing the word 'bucket.' Token bucket allows bursts up to a capacity and rejects or delays only the excess, prioritizing throughput and burst tolerance. Leaky bucket forces a constant output rate regardless of input burstiness by queueing excess requests, prioritizing smoothness and predictability for the downstream system at the cost of added latency.

Most use a shared, low-latency data store — commonly Redis — to hold each client's token count or request timestamps, with atomic increment-and-check operations (e.g., Lua scripts in Redis) to avoid race conditions where two servers both check the counter before either updates it. Some systems trade perfect global accuracy for lower latency by using local approximate counters that periodically synchronize, accepting slightly looser enforcement in exchange for not adding a network round-trip to every request.

HTTP 429 (Too Many Requests) tells the client its request was rejected specifically due to rate limiting rather than a server error or bad input. The accompanying Retry-After header (in seconds, or as a timestamp) tells a well-behaved client how long to wait before retrying, which lets clients implement backoff correctly instead of guessing or retrying immediately and making the overload worse.

By assuming request arrivals are roughly uniform within a window, it can approximate the true rolling count from just the previous window's total and the current window's running total, weighted by how far into the current window the clock has moved. This trades a small amount of accuracy for far less memory than a sliding window log, which must store one timestamp per request.