Infrastructure
Rate Limiting
Rate limiting controls the rate of requests a client can send to a service. It protects against abuse, ensures fair resource usage, and maintains system stability under load.
- Protection â Prevent overload and denial-of-service attacks
- Fairness â Ensure equitable resource allocation
- Stability â Maintain predictable performance under load
Rate limiting is the first line of defense for any production service.
Why Rate Limit?
Uncontrolled traffic can cascade failures through a system. Rate limiting provides a controlled degradation path.
Types of Limits
| Limit Type | Description | Example |
|---|---|---|
| Per-client | Limit per individual user/IP | 100 requests/minute per user |
| Global | System-wide limit | 10,000 requests/second total |
| Per-endpoint | Limit per API endpoint | 10 writes/second per user |
| Tiered | Different limits per plan | Free: 100/min, Pro: 1000/min |
Token Bucket
The most widely used rate limiting algorithm.
Sliding Window Log
Tracks exact timestamps of each request in a sorted set.
Sliding Window Counter
A hybrid approach combining fixed windows with interpolation.
Fixed Window
The simplest approach: count requests in fixed time periods.
Distributed Rate Limiting
Rate limiting across multiple service instances.
Distributed Challenges
| Challenge | Solution |
|---|---|
| Race conditions | Use atomic operations (Redis INCR with EXPIRE) |
| Network latency | Local rate limiting with periodic sync |
| Redis failure | Fallback to local limiting or fail open |
| Clock skew | Use logical timestamps or server-side time |
Algorithm Comparison
| Algorithm | Accuracy | Memory | Burst Handling | Complexity |
|---|---|---|---|---|
| Token Bucket | High | O(1) | Allows bursts up to C | Low |
| Sliding Window Log | Exact | O(n) per client | No bursts | Medium |
| Sliding Window Counter | Approximate | O(1) | Smooth | Low |
| Fixed Window | Low | O(1) | Boundary bursts | Very Low |
| Leaky Bucket | High | O(1) | Smooth output | Low |
Practice Exercises
-
Design: Design a rate limiting system for an API gateway handling 1M requests/second with per-user limits of 100 requests/minute. Include Redis architecture and failover.
-
Analysis: Compare token bucket and sliding window counter for a video streaming service that allows 10-second bursts of high bitrate.
-
Distributed: Implement distributed rate limiting using Redis with a Lua script. Handle the case where Redis is unavailable.
-
Edge Case: A user has a rate limit of 10 requests/minute. They make 10 requests at 12:00:59 and 10 more at 12:01:01. How does each algorithm handle this?
What to Learn Next
-> Proxy and Reverse Proxy Forward proxy, Nginx, HAProxy, and SSL termination.
-> API Design REST, GraphQL, gRPC, and API gateway patterns.
-> CDN Edge caching, DNS routing, and content distribution.
-> Load Balancing Distribution algorithms and L4 vs L7 load balancing.
-> Security Patterns Authentication, authorization, encryption, and mTLS.
-> Observability Logging, metrics, tracing, and monitoring.