A rate limiter can only bucket by what the request carries when it runs, so choosing a key is choosing a stage in the pipeline. Early there is nothing but a network address. Later there is a credential the request asserts before anything verifies it. The ticket saying an entire office gets 429 comes out of that ordering.
A limiter keys on whatever the request carries at that line
A limiter is middleware, and middleware has a position. Everything about which keys are available to it follows from where it sits, because a key has to be derived from facts the request has already established at that line.
At the front door, the request has established very little. There is a body nobody has validated, headers anybody can write, and the address the packets arrived from. Of those, only the address is outside the caller’s control, and even that is a weaker fact than it looks. A credential is the better subject, but only once something has read it, and middleware runs ahead of the code that does the reading. What a limiter usually holds is a credential the caller claims, and a claim costs nothing to manufacture.
| What the request has established at that line | What a key can be | Who shares the bucket |
|---|---|---|
| Packets arrived from an address | The address, normalized | Everyone behind that egress, and on IPv6 everyone in the same /56 |
| A credential was presented, and nothing has read it | A digest of the string presented | Whoever sends that exact string |
| A credential was read and rejected | The address, again | Everyone behind that egress |
The middle row is weaker than it looks and the third row is awkward by construction. The last two sections are about how they hold each other up.
Before identity, the only subject is an address
Take a public endpoint a visitor’s browser calls, where the request names a project by a key that is published in page source. The cap on it has to run before that key is resolved against the database, because the database read performed for an unauthenticated stranger is part of what the cap is protecting. So the limiter sits ahead of every lookup, and the address is all it has.
Using the address raw does not work, in two separate ways.
The first is normalization. IPv4 passes through unchanged, an IPv4 address written in IPv6 form is
rewritten back to its IPv4 spelling, and an IPv6 address is truncated to its /56 prefix. Skip
that step and an IPv6 client gets a fresh bucket for every request by walking to the next address
in a range it already holds, which is the whole cap gone rather than a cap slightly off.
The second is that we do not want to keep the address at all. The key is an HMAC of it under a
salt, so the store holds a digest and nothing reversible. That is what forces the order: the digest
destroys the structure the /56 collapse needs, and there is no way to normalize a hash afterwards.
Normalize, then HMAC. Both orders produce a key that looks perfectly reasonable in a debugger, and
only one of them caps anything.
The property this buys is worth naming, because it is the reason to accept a coarse key at all: a caller cannot escape the bucket by rotating the public project key, since the limiter has not read it yet.
Why the whole office gets 429
Two hundred people behind one corporate egress are one address, and therefore one bucket. Nothing about the limiter is malfunctioning when the ticket arrives. It answered a question about the only subject it had, and that subject happens to be a building.
The same shape shows up wherever addresses are shared: a VPN concentrator, carrier-grade NAT on a
mobile network, a CI runner pool, a university. IPv6 does not rescue it either. A limiter that
buckets by /56 is treating a whole subscriber site as one caller, which is the correct reading
given that RFC 6177 expects an end site to receive a
prefix of roughly that size rather than a single address.
Once that is clear, the fix stops being a search for a cleverer key. There is no better subject in the request yet. What is left is to size the shared cap for a building rather than for a person, and to put the tight cap on the credential, where a bucket belongs to one caller and a noisy neighbour cannot spend somebody else’s allowance. That is where the next section starts, and it is where the interesting problem is, because a credential is a fact the request asserts before anybody has checked it.
A bucket per token, before anything has read the token
Our MCP endpoint allows 240 calls a minute per token, which is the ceiling the developer reference publishes, and the bucket is a SHA-256 digest of the bearer token carrying a namespace prefix so a credential key can never collide with an address key in the same store.
It is worth being exact about where that runs. Both limiters on the endpoint are mounted ahead of
the JSON body parser and ahead of the handler that resolves a token against the stored hashes, so
the key function only ever reads a header: it takes whatever follows Bearer in Authorization,
hashes it, and returns. Nothing has compared that string to anything.
So the bucket belongs to the token presented, never to a token somebody accepted, and by itself that caps very little. A caller sending a fresh random bearer on every request opens a fresh 240-a-minute bucket every time, because a different string is a different key. A key the caller chooses is not a ceiling.
What turns it into one is the limiter standing in front of it. The failure limiter is mounted first, it keys on the address, and it charges every request that does not come back under 400. An invented token opens its own generous bucket on the per-token limiter and collects a 401 from the handler behind it, and that 401 is billed back to the address. Manufacturing credentials buys allowance under a key the caller controls and spends allowance under the one key they do not, which is why the pair caps what neither half caps alone.
The digest is a storage decision, not a security control. Rate limit keys live for the length of the window, get logged, and turn up in metrics labels, so keying on the raw token would create a second copy of every credential in places nobody threat models. Bucketing behaves identically either way.
The ceiling is high on purpose. An agent working through a real triage session is bursty and legitimate, and the limit exists to cap what a leaked token can drain rather than to pace ordinary use. What the per-token key buys, read together with the limiter in front of it, is isolation between real holders: two valid tokens are two buckets, so one client having a bad day cannot take anybody else’s allowance, which is exactly the property the address stage cannot offer.
One line in that key function decides the rest. If there is no bearer token at all, the key falls back to the address. An unauthenticated caller is therefore back at stage one, sharing the building’s bucket with everybody else who has not identified themselves. The published per-token ceiling starts applying the moment a token is presented, which is a different moment from the one it is honoured, and that gap is the same lesson the ladder of reporter identity provenance teaches from the other end of the pipeline: what a value proves is a separate question from what it is worth.
The limiter that cannot wait for identity
Now the third row of that first table. A limiter whose job is to bound credential guessing has a problem no key choice solves: it fires precisely when the credential turned out to be wrong. Keying on the credential would hand a guesser a brand new bucket with every wrong guess, which counts attempts and stops none of them. So it keys on the address, sits at stage one, and inherits the office problem by construction.
The way out is a refund rather than a better key. Everything that does not come back under 400 is counted, which takes in 5xx as well as 4xx, and everything below that line is refunded once the response has gone out. A caller who authenticates successfully is therefore never charged against the shared bucket at all. Failures accumulate; working traffic does not.
Collateral damage falls sharply and does not vanish. Sixty failed requests a minute is the entire budget, and it belongs to your egress rather than to you: one client in your office looping on an expired token can spend all of it, and the next colleague through the door is refused while holding a perfectly good credential. The limiter has to decide before it knows whether that credential was any good, which is the same constraint from the top of this post arriving at its most expensive form. On our side the address does not survive that decision as an address: our security practices keep it as a one-way salted hash, if at all.
So the question worth asking about a limiter is how far down the pipeline it sits, because that position has already decided which of its keys are honest ones. A key can always name something the request has not proved yet; what it cannot do is make the naming true. Either move the limiter to where the proof is, or put a second one in front of it that charges for the claim.
