System Design β Data Systems
Caching Strategies
Caching is the single most impactful performance optimization in system design. By storing frequently accessed data in fast storage (memory), you can reduce latency from milliseconds to microseconds and dramatically reduce database load.
- Cache-Aside β Application manages cache explicitly
- Write-Through β Writes go to cache and database simultaneously
- Redis vs Memcached β When to use each caching technology
There are only two hard things in computer science: cache invalidation and naming things. β Phil Karlton
What Is Caching?
Where Caches Live
Cache Topologies
| Property | Local Cache | Distributed Cache |
|---|---|---|
| Latency | Nanoseconds | Microseconds-milliseconds |
| Scope | Single process | All instances |
| Invalidation | Simple | Complex (broadcast needed) |
| Memory | Limited by process | Scales independently |
| Consistency | Eventual | Configurable |
Cache Eviction Policies
When the cache is full, eviction policies determine which entries to remove:
| Policy | Description | Trade-off |
|---|---|---|
| LRU | Least Recently Used | Good general-purpose, may evict rarely-used but important items |
| LFU | Least Frequently Used | Better for skewed distributions, slower to update |
| FIFO | First In First Out | Simplest, no frequency/recency awareness |
| TTL | Time To Live | Expires after fixed duration, may evict useful data |
| Random | Random eviction | Surprisingly effective, no tracking overhead |
Cache-Aside (Lazy Loading)
The most common caching pattern:
Flow:
- Application checks cache for data
- Cache hit: Return data directly from cache
- Cache miss: Fetch from database, store in cache, return data
- On write: Update database, invalidate cache (don't update cache)
Write-Through
Writes go to both cache and database simultaneously:
Pros: Cache is always consistent, simple to implement Cons: Write latency is cache + DB, cache may be filled with data never read
Write-Back (Write-Behind)
Writes go to cache first, then asynchronously to database:
Pros: Very fast writes, batched database writes Cons: Data loss risk if cache fails before flush, complex implementation
Redis vs Memcached
| Feature | Redis | Memcached |
|---|---|---|
| Data structures | Strings, lists, sets, hashes, sorted sets, streams | Strings only |
| Persistence | RDB + AOF | None |
| Replication | Built-in primary-replica | None |
| Cluster mode | Redis Cluster | Client-side sharding |
| Memory efficiency | Higher overhead per key | Very memory efficient |
| Pub/Sub | Built-in | None |
| Lua scripting | Supported | Not supported |
| Best for | Complex data, persistence, pub/sub | Simple key-value caching |
Cache Patterns in Practice
Cache Stampede Prevention
When a popular cache entry expires, many requests simultaneously try to rebuild it:
Cache Invalidation Strategies
| Strategy | Description | Consistency |
|---|---|---|
| TTL-based | Expires after fixed duration | Eventual |
| Event-driven | Invalidate on write events | Near real-time |
| Version-based | Cache key includes version | Strong |
| Tag-based | Group related keys for bulk invalidation | Configurable |
Practice Exercises
-
Design: Design a caching strategy for a news website with 10M daily visitors. What would you cache? What TTL would you use? How do you handle breaking news?
-
Analysis: Compare cache-aside, write-through, and write-back for a banking application. Which pattern is most appropriate and why?
-
Debugging: A system has a 99% cache hit ratio but still shows high latency at p99. What could cause this? How would you diagnose it?
-
Architecture: Design a distributed caching layer for a global e-commerce platform. Consider: data locality, consistency, failure handling, and warming the cache.
What to Learn Next
-> Databases SQL vs NoSQL, indexing, replication, and sharding.
-> Load Balancing Algorithms, health checks, and L4 vs L7.
-> Message Queues Kafka, RabbitMQ, event-driven architecture.
-> CAP Theorem Consistency models, availability, and partition tolerance.
-> Microservices Service decomposition, discovery, and API gateways.
-> Scalability Fundamentals Vertical vs horizontal scaling and capacity planning.