Rate Limiter System Design

Problem

noisy refused over quota ≥ 1800 rps

honest error rate < 1%

No endpoint takes more than its own limit, even from many clients that are each under quota.functional · not simulated

api requests in < 50000 rps

Limiting keeps working while the counter store is unreachable, and the API stays up.functional · not simulated

When an expensive endpoint slows down, no single client can hold all of its capacity.functional · not simulated

When the API is simply full, background work is dropped before any user-facing request.functional · not simulated

throughput ≥ 49000 rps

clients × pods → store calls and memory

Clients10,000

Peak requests/s50,000

Gateway pods10

Sync interval (ms)100

Quota per client (requests/s)100

Results appear once the simulation starts.

p99 latency

—ms

error rate

—%

throughput

—rps

refused over quota

—rps

spent so far

—1 min = 1 mo

Live metrics, last 60 seconds
Metric	Now	Versus baseline	60 s low	60 s high
p99 latency	—	—	—	—
error rate	—	—	—	—
throughput	—	—	—	—
refused over quota	—	—	—	—

Loading the simulation…