← All articles

Rate limiting with token buckets, windows and one place to count

Updated Nov 8, 2026 · 3 min read

A rate limiter caps how many requests one client may make in a period of time. It protects a service from scrapers, runaway scripts and honest bugs, and it keeps one noisy customer from slowing everyone else down. It sits in front of the real work, usually in an API gateway, and answers one question per request. Does this one get through?

Who counts as one client

Limits are kept per key. For a public API that is usually the API key, because it names a customer and their plan. Without one, you fall back to the user id or the IP address. IPs are a weak key, because a whole office can share one and an attacker can rotate through thousands.

Saying no politely

A request over the limit gets 429 Too Many Requests. Add a Retry-After header with the seconds until the client may try again, and well-behaved clients will back off on their own. Many APIs also send RateLimit-Remaining on every response so clients can slow down before they hit the wall.

The four common algorithms

AlgorithmHow it worksGood atWatch out for
Fixed windowOne counter per key per minute, reset on the minuteCheapest, with one number per keyDouble bursts at the edge of each window
Sliding window logKeep a timestamp for every request in the last minuteExactMemory grows with traffic
Token bucketTokens refill at a steady rate up to a cap; each request takes oneAllows short bursts, holds the averageTwo numbers to tune
Leaky bucketRequests queue and leave at a steady ratePerfectly smooth outputBursts wait instead of failing fast

The edge problem with fixed windows

With a limit of 100 a minute and a counter that resets on the minute, a client can send 100 requests at 12:00:59 and another 100 at 12:01:00. That is 200 requests in two seconds, twice what the limit was meant to allow. Sliding windows fix this by always looking back a full minute from now.

Sliding window100
Fixed window, across the reset200
Most requests one client can get through in two seconds, with a limit of 100 a minute.

Why token buckets are the default

Real clients are bursty. A web app loads a page and fires 30 requests at once, then goes quiet. A token bucket with room for 30 tokens, refilled at 10 a second, lets the whole burst through immediately and still holds the client to 10 a second over time. It only needs two numbers per key, the tokens left and when it last refilled.

tokens = min(capacity, tokens + (now - last) × rate)
last = now
if tokens >= 1: tokens -= 1, allow
else: reject with 429

Where to count

Most services run several gateway servers behind a load balancer. If each one counts in its own memory, a client whose requests are spread across 8 gateways gets 8 times its limit. The count has to live in one shared place, usually Redis, where an atomic script reads and updates the bucket in one step so two gateways cannot both spend the last token.

Clientone API keyLoad balancerGateway 1Gateway 2Redisone bucket per key
Every gateway checks the same bucket, so the limit holds however the load balancer spreads requests.

Failing open or closed

If the limiter’s store goes down, you must choose. Failing open lets every request through, so customers keep working but you are unprotected. Failing closed rejects everything. It’s safe, but it turns a limiter outage into a full outage. Most public APIs fail open and alert loudly.