🎓 A1Learner
🔍
0 of 42 lessons completed

Rate Limiter: Trade-Offs and Wrap-Up

Failing open lets requests pass when the limiter itself is down, and failing closed blocks them instead. This lesson covers the trade-offs and wraps up the rate limiter design.

Fail open and fail closed, side by side

The limiter is down, and the same arrivals face each policy in turn. The printout shows the full gap between the two choices.

arrivals = ["ana", "ben", "ana", "cy", "ben"]

for mode in ["fail open", "fail closed"]:
    passed = 0
    for user in arrivals:
        if mode == "fail open":
            passed += 1
    print(mode, "passed", passed, "of", len(arrivals))

The refusal carries a wait hint

Requests under the limit pass with a normal response. The request over the limit gets the refusal response plus a hint for when to try again.

limit = 3
count = 0

for i in range(1, 5):
    count += 1
    if count <= limit:
        print("request", i, "status 200")
    else:
        print("request", i, "status 429, retry after 30 seconds")

One heavy user cannot crowd out the rest

One user sends past the personal limit while another sends a little. The heavy user is refused, and the lighter user still passes in full.

limit = 3
counts = {}
arrivals = ["ana", "ana", "ana", "ana", "ana", "ben", "ben"]

for user in arrivals:
    counts[user] = counts.get(user, 0) + 1
    print(user, "accepted" if counts[user] <= limit else "rejected")

print("ana count", counts["ana"])
print("ben count", counts["ben"])

Example

limit = 3
counts = {}
arrivals = [("ana", False), ("ana", False), ("ana", False), ("ana", False), ("ana", True), ("ben", True)]

for user, limiter_down in arrivals:
    if limiter_down:
        print(user, "limiter down, passed by fail open")
        continue
    counts[user] = counts.get(user, 0) + 1
    if counts[user] <= limit:
        print(user, "status 200")
    else:
        print(user, "status 429, retry after 30 seconds")

print("ana counted", counts.get("ana", 0))

Key points

  • Fail open: requests pass unchecked while the limiter is down, so the service stays reachable without its guard.
  • Fail closed: requests are blocked while the limiter is down, so the guard holds and the service takes the outage.
  • Cost of failing open: an outage that lasts 2 minutes can let a single user send 200 requests past a failed limiter.
  • Status 429: the response that means too many requests, sent when a user passes the limit.
  • Retry-After: a hint in the 429 response that says how many seconds to wait before trying again.
  • Fairness: a per user limit keeps one heavy user from crowding out every other user.

In this example, the finished limiter counts in a shared per user tally, refuses the over-limit request with a wait hint, and then fails open during a scripted outage. The wrap-up shape is the full interview run, from the clarified ask to the trade-off call.

Note

the finished design clarifies the ask, checks each request in the gateway, counts in one shared store, and answers over-limit requests with a clear refusal.

Try it Yourself

Edit the code, then press Run.

Running is simulated

Exercise

The limiter itself is down and the design fails open. What happens to incoming requests?