Backend & Data
Token bucket vs leaky bucket — how rate limiters work
Every API that says "60 requests per minute" needs an algorithm underneath deciding exactly which requests to allow and which to reject. Two designs dominate, and the difference between them is entirely about bursts — whether you can spend your whole budget at once or must trickle it out. Meet the token bucket and the leaky bucket.
Token bucket — save up, then burst
Picture a bucket that fills with tokens at a steady rate — say one per second, up to a cap. Each request must take a token to proceed; no token, it's rejected (or waits). The magic is the cap: if you've been idle, tokens accumulate, so you can fire a burst of requests all at once, then throttle back to the refill rate. It allows spikes while still capping the long-run average — which is why it's the most popular choice for public APIs.
Leaky bucket — a steady drip
Now picture requests pouring into a bucket with a hole in the bottom that leaks at a fixed rate. Output is perfectly smooth no matter how spiky the input — the bucket absorbs a burst up to its capacity, then anything more overflows and is dropped. It trades burst-friendliness for a smooth, predictable outflow, which is ideal when the thing downstream can only handle a steady pace.
Token bucket: you earn tokens over time and can spend a pile at once. Leaky bucket: requests drain at a constant drip and extras spill over. One rewards saving up; the other enforces a steady pace.
Which, and where it fits
- Token bucket when occasional bursts are fine and you care about the average — most user-facing and API rate limits.
- Leaky bucket when the downstream needs a constant, smooth rate and bursts would hurt it.
Whichever the server uses, it's the machinery behind the 429 Too Many Requests you meet as a client — so your side pairs it with backoff to retry politely, and a circuit breaker when the far side is clearly overwhelmed. Same problem — controlling flow — solved from both ends.