Rate limiting with a token bucket
Your API in front of the model should accept short bursts without letting anyone hog it. A token bucket does both.
Write createBucket({ capacity, refillPerSec, now }) that returns an object with tryTake().
The bucket starts full with capacity tokens. tryTake() spends one token and returns true, or returns false when no whole token is left. Tokens refill at refillPerSec per second (fractions too), never above capacity.
Always read the time with now() (in milliseconds): that is how the challenges use a fake clock. You already have makeClock(), which returns { now, advance(ms) }.
Challenges 0/4
- Starts full and runs dry
- Refills one token per second
- Never goes above capacity
- Accumulates fractions of a token
function createBucket({ capacity, refillPerSec, now = () => Date.now() }) {
let tokens = capacity;
return {
tryTake() {
// add (elapsed seconds * refillPerSec) tokens, capped at capacity
if (tokens >= 1) { tokens -= 1; return true; }
return false;
},
};
}
const clock = makeClock();
const bucket = createBucket({ capacity: 2, refillPerSec: 1, now: clock.now });
console.log(bucket.tryTake(), bucket.tryTake(), bucket.tryTake());
clock.advance(1000);
console.log('after 1 s:', bucket.tryTake());Console output appears here (console.log).
Go deeper: the related guide →
This in production, with your data? Let's talk for 15 minutes →