Most teams still meter the wrong thing
Rate limiting fails when it treats every request as equal and every source IP as a durable identity. Attackers already rent exit space, rotate through compromised set-top boxes, and spray cheap guesses across auth and business APIs. A single global counter per address looks tidy in a dashboard and still leaves your expensive endpoints open.
Recent pressure makes this concrete. Industry takedowns of botnets that turned more than a million TV boxes into residential proxies showed how quickly a campaign can present as thousands of low-volume clients. Parallel waves of PHP and gateway exploit scouting, credential stuffing after breach dumps, and AI-assisted probing all share the same pattern: each exit knocks lightly while the fleet as a whole drains capacity and burns analyst time.
Why shared buckets keep shipping
Platform teams inherit defaults that favor fairness over risk. A reverse proxy or API gateway ships with a fixed requests-per-minute rule keyed on client IP. Product owners ask for one number that applies to the whole surface. Security gets a green checkbox once that number exists. Nobody owns the harder question of which routes cost the most to serve, which ones mint sessions or change state, and which ones attackers need for ransomware staging or account takeover.
Shared buckets also flatter monitoring. Spike charts look clean when one threshold trips. After a TfL-scale consumer data incident or a fresh credential dump, helpdesk lockouts and MFA fatigue obscure the real campaign. By the time someone notices that password-reset and token-refresh routes absorbed most of the volume, the spray has already mapped which accounts still accept basic auth or weak second factors.
Another trap is treating authenticated traffic as trusted. Stolen OAuth tokens, partner SSO sessions, and long-lived API keys ride inside the same generous authenticated quota that legitimate integrations need. Prestige or government-looking source addresses, including temporary rentals of high-profile ranges, further weaken IP reputation as a sole throttle key.
Design limits around cost and intent
Effective API rate limiting is a control plane for abuse economics, not a polite queue. Start by classifying routes into cost tiers and abuse intent, then bind distinct limiters to each tier.
Tier the surface before you pick algorithms
- Identity and session minting: login, password reset, MFA challenge, OAuth authorize and token endpoints, API key creation.
- High-cost reads: search, export, report generation, large list pagination, LLM or enrichment gateways.
- State-changing writes: transfers, privilege changes, webhook registration, admin CRUD.
- Cheap health and metadata: readiness probes, public docs, static version endpoints.
Give identity and write routes the tightest ceilings and the richest keys. Leave headroom on cheap health checks so monitoring does not starve during an incident.
Choose keys that survive exit rotation
IP-only keys collapse when residential proxies and carrier NAT recycle addresses. Combine dimensions:
- Unauthenticated callers: IP prefix or ASN bucket plus User-Agent class plus route family.
- Authenticated callers: subject or client_id first, then IP as a secondary burst cap.
- Partner integrations: signed client identity with a separate fair-share pool so one noisy tenant cannot exhaust the shared edge.
- Admin portals: geo severity tier plus device or session binding, with overnight windows shrunk when consumer VPN and set-top exits spoil country labels.
When threat intel reports ASN abuse velocity or a set-top proxy family takedown, feed those signals into quarantine multipliers on the edge rather than waiting for single-IP blocks to go stale.
Match the algorithm to the failure mode
- Token bucket fits bursty legitimate clients that need short spikes and a steady refill. Use it for partner APIs with documented SLA bursts.
- Sliding window log or counter fits auth endpoints where attackers probe just under fixed minute boundaries. Prefer it for login and reset routes.
- Leaky bucket / queue fits expensive exports and report jobs where you must shape concurrency, not only count hits.
- Cost-based or weight-based limits charge search, export, and AI inference more tokens per call so a crafty client cannot burn CPU under a naive request cap.
Store counters in a shared store near the edge (Redis, gateway native stores, or mesh rate-limit filters) with short TTLs and deterministic key templates. Document clock skew and multi-region behavior so failover regions do not silently double the allowance.
A playbook you can run this quarter
1. Inventory and score routes
Export your OpenAPI or gateway route table. Tag each path with auth requirement, side effects, typical latency, and dependency cost. Rank the top twenty by abuse value: anything that returns PII, mints credentials, or triggers outbound mail or SMS belongs in the first remediation wave.
2. Split anonymous and authenticated policies
Publish two policy sets. Anonymous traffic gets low ceilings and progressive challenges (CAPTCHA, proof-of-work, or stepped delays) after soft thresholds. Authenticated traffic gets per-principal quotas with a hard global safety cap. Never let a valid token inherit the anonymous IP bucket alone.
3. Add progressive response, not only hard 429
Sequence controls so operators keep signal:
- Emit rate-limit headers early so good clients back off.
- Return 429 with Retry-After for clear overuse.
- Downgrade to challenge or shadow-deny for identity routes under spray.
- Quarantine ASN or prefix cohorts when velocity matches known proxy-fleet patterns from recent botnet disclosures.
Log the decision reason code in a field analysts can join to SIEM hunts the same hour. Tie those codes to playbooks for ransomware scout weeks and PHP mass-scan advisories so on-call knows whether to patch, challenge, or quarantine.
4. Rehearse under live-shaped load
Before the next advisory crowd hits your edge, run a tabletop that mixes three traffic shapes: low-and-slow credential guesses from many exits, a burst against one expensive export route, and a noisy health-check flood. Measure auth latency, queue depth, and false lockouts. Budget VPN portal and partner SSO failures per exit cluster so set-top proxy rotation does not look like a regional outage.
5. Close the loop with intel without drowning the limiter
Threat intelligence earns its keep when it changes multipliers and keying, not when it pastes raw IOC floods into the deny list. After a large proxy-fleet bust, rank callback persistence and ASN membership higher than singleton IPs. After a gateway RCE notice, temporarily tighten unauthenticated admin and management API routes on hosting ASNs that spike in your honeypots. Keep a human gate on automated quarantine expansion so AI-generated scout noise does not lock out a whole customer vertical.
Implementation details that hold up in production
Put enforcement at the earliest trusted hop that sees normalized identity: edge gateway for anonymous and TLS-terminated routes, service mesh or app middleware once the caller principal is known. Duplicate critical auth limits in both places during migration so a misconfigured bypass path cannot skip the bucket.
Standardize headers: RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset, and Retry-After. Include a stable policy-id in logs and, when safe, in error payloads so support and detection speak the same language.
Watch these metrics weekly:
- 429 and challenge rates by route tier and by ASN
- Median and p95 auth latency while limits are active
- Share of authenticated 429s caused by a single client_id
- Correlation of honeypot hits with production limit trips during exploit scout waves
- Time from intel-driven multiplier change to measurable drop in lockouts
For multi-tenant SaaS, allocate fair-share tokens per tenant with a small global emergency brake. For internal admin APIs, prefer session-bound ceilings and geo severity tiers over wide office-IP allowlists that remote consulting exceptions slowly erode.
What good looks like on the next abuse week
When ransomware groups raise scanning and credential reuse in the same week as consumer-proxy rotation, your identity routes shed load through sliding-window limits and challenges while partner batch jobs continue under token buckets. Analysts receive Monday hunting packets that already list which policy-ids fired, which ASNs absorbed quarantine multipliers, and which API keys need rotation. Leadership sees fewer helpdesk lockout spikes and a clearer link between edge decisions and account-takeover risk.
Rate limiting then stops being a single number on a wiki page. It becomes a set of deliberate economic controls aligned to how APIs are actually abused: distributed exits, stolen principals, and expensive routes that attackers need more than your homepage.