Build a rate limiter

Q: When is an interface worth introducing?

When the second implementation arrives, and not one minute earlier. A Protocol with one implementation is a layer a reviewer has to read through to reach the code that does the work, and in this round it reads as over-engineering. The moment a second policy exists, the same Protocol pays for itself: the limiter holds a dict of them and never learns which is which. Say the rule out loud while you are not writing it, because declining an abstraction on purpose scores better than never noticing it.

Q: What do I say when they ask about many servers?

Name what breaks first: per-process state means N servers allow up to N times the limit, because each one counts only the calls it saw. Then give one design and one trade-off. A shared store such as Redis, with the count and the expiry held there, makes the limit global at the cost of a network hop on every call and a decision about what to do when the store is unreachable. Fail open and you stop limiting during an outage; fail closed and an outage in the limiter takes down the API it was protecting.

One spec, three stages, fifty minutes: a sliding-window log, then a limit per user, then a second policy behind the first interface the problem actually earns.

Should I write the sliding-window log or the token bucket first?

The log, almost always. It follows straight from the words of the spec, which are usually some form of "fewer than N calls in the last W seconds", so a reviewer can check it against the sentence they gave you. The token bucket answers a different question, "how fast on average, and how big a burst", and if you reach for it first you have quietly changed the requirement. Write the log, get it passing, and say out loud that the bucket exists and costs constant memory, which is the sentence that earns the follow-up.

Where should the clock come from?

From a parameter, passed in when the limiter is built. Any call to time.time() inside the logic means a test of expiry has to either sleep for real seconds or monkey-patch the standard library, and an interviewer watching you do the second one has learned something about how you write code. A clock parameter is one word in the signature and it makes every time-dependent test instant and exact. Do it in the first stage, before anything needs it, because the first test you write will need it.

Is a call exactly W seconds old still inside the window?

That is a question for the interviewer, not a decision to make silently, and asking it is worth a mark. The common answer is no: the window covers the half-open range from now minus W, exclusive, to now, inclusive, which means a call exactly W seconds old has just left. That choice turns into one character of code, <= against <, and into one test that pins it. Write that test in the first stage and it will still be guarding you three refactors later.

When is an interface worth introducing?

When the second implementation arrives, and not one minute earlier. A Protocol with one implementation is a layer a reviewer has to read through to reach the code that does the work, and in this round it reads as over-engineering. The moment a second policy exists, the same Protocol pays for itself: the limiter holds a dict of them and never learns which is which. Say the rule out loud while you are not writing it, because declining an abstraction on purpose scores better than never noticing it.

What do I say when they ask about many servers?

Name what breaks first: per-process state means N servers allow up to N times the limit, because each one counts only the calls it saw. Then give one design and one trade-off. A shared store such as Redis, with the count and the expiry held there, makes the limit global at the cost of a network hop on every call and a decision about what to do when the store is unreachable. Fail open and you stop limiting during an outage; fail closed and an outage in the limiter takes down the API it was protecting.