🎓 A1Learner
🔍
0 of 42 lessons completed

Design a Rate Limiter: The Ask

Rate limiting is a rule that caps how many requests one user may make in a set time. This lesson clarifies the ask, including who is limited, where the limiter sits, and what happens on rejection.

Each user gets a separate count

The gateway keeps one count per user. One user going over the limit does not change the count of any other user.

limit = 3
counts = {}
arrivals = ["ana", "ana", "ana", "ana", "ben", "ben"]

for user in arrivals:
    counts[user] = counts.get(user, 0) + 1
    if counts[user] <= limit:
        print(user, "request allowed, count", counts[user])
    else:
        print(user, "request refused, count", counts[user])

The gateway stands in front

Every request passes the gateway first. A refused request stops there, so the app server only ever sees allowed requests.

limit = 3
counts = {}
app_log = []
arrivals = ["ana", "ana", "ana", "ana", "ben"]

for user in arrivals:
    counts[user] = counts.get(user, 0) + 1
    if counts[user] <= limit:
        app_log.append(user)
        print(user, "passed the gateway")
    else:
        print(user, "stopped at the gateway")

print("app server saw", len(app_log), "requests")

The scope picks what gets counted

A limit can follow a user, an address, or a key. Each scope keeps its own separate count, as the printout shows.

arrivals = [("user", "ana"), ("user", "ana"), ("address", "203.0.113.10"), ("key", "key-7"), ("user", "ana")]
counts = {}

for scope, name in arrivals:
    label = scope + ":" + name
    counts[label] = counts.get(label, 0) + 1
    print(label, "count is now", counts[label])

Example

limit = 3
counts = {}
app_counts = {}
arrivals = ["ana", "ben", "ana", "cy", "ana", "ben", "ana", "cy"]

for user in arrivals:
    counts[user] = counts.get(user, 0) + 1
    if counts[user] <= limit:
        app_counts[user] = app_counts.get(user, 0) + 1
        print(user, "request passed")
    else:
        print(user, "request refused")

for user in ["ana", "ben", "cy"]:
    print(user, "reached the app server", app_counts.get(user, 0), "times")

Key points

  • Worked limit: each user may make 100 requests per minute in this design.
  • Two users at the limit together account for 200 requests in one minute.
  • API gateway: the single entry point that receives every request before it reaches the app servers.
  • Per user: the limit follows the signed-in user, so one busy user cannot spend another user's share.
  • Per address: the limit follows the network address when no user is signed in.
  • Per key: the limit follows an access key that an app uses to call the service.
  • Rejection: a request over the limit gets a clear refusal and never reaches the app servers.

In this example, the gateway in front of the app servers checks each arrival against a small demo limit and keeps a separate count per user. Only the busiest user is refused, and the final counts show exactly what each user sent through.

Note

the limiter sits in the gateway, in front of the app servers, so every request is checked in one place.

Try it Yourself

Edit the code, then press Run.

Running is simulated

Exercise

A service wants one busy user to be unable to block everyone else. Which scope fits best?