System Design Foundations
Scalability Fundamentals
Scalability is the ability of a system to handle increased load by adding resources. This guide covers the mathematical foundations, strategies, and trade-offs for building systems that grow gracefully with demand.
- Vertical Scaling — Upgrade the machine (bigger CPU, more RAM)
- Horizontal Scaling — Add more machines to distribute load
- Capacity Planning — Predict and prepare for future growth
Scale is not about handling today's traffic—it's about handling tomorrow's.
What Is Scalability?
Scalability is a system's ability to maintain or improve performance as resources (compute, memory, storage, network) are added.
Vertical vs Horizontal Scaling
| Dimension | Vertical | Horizontal |
|---|---|---|
| Complexity | Low (no code changes) | High (distributed systems) |
| Ceiling | Limited by largest machine | Virtually unlimited |
| Cost | Exponential growth | Linear growth |
| Fault Tolerance | Single point of failure | Redundant |
| Latency | No inter-node communication | Network overhead |
The Math of Scaling
Amdahl's Law and Scaling
When you add N machines, the theoretical maximum speedup is:
Gustafson's Law
Gustafson's Law offers a more optimistic view—when you add resources, you can solve larger problems:
Load Balancing
Load balancing distributes incoming requests across multiple servers to ensure no single server becomes a bottleneck.
Common Load Balancing Algorithms
| Algorithm | How It Works | Best For |
|---|---|---|
| Round Robin | Distributes requests sequentially | Servers with equal capacity |
| Weighted Round Robin | Distributes proportionally to weight | Servers with different capacities |
| Least Connections | Routes to server with fewest active connections | Long-lived connections |
| IP Hash | Maps client IP to a specific server | Session persistence needs |
| Least Response Time | Routes to server with lowest latency | Latency-sensitive workloads |
| Consistent Hashing | Maps requests to servers using hash ring | Minimizes redistribution on scaling |
Capacity Planning
Capacity planning ensures your system can handle future load without over-provisioning.
Step-by-Step Capacity Planning
- Forecast demand: Estimate future traffic based on growth rates
- Profile current system: Measure resource utilization at current load
- Identify bottlenecks: Which resource (CPU, memory, I/O, network) is the limiting factor?
- Calculate headroom: How much capacity is needed for the forecast?
- Plan scaling triggers: At what utilization threshold should you scale?
Scaling Strategies
The Database Bottleneck
The most common scaling bottleneck is the database. Strategies include:
- Read replicas: Offload read traffic to replica databases
- Sharding: Partition data across multiple database instances
- Caching: Store frequently accessed data in memory
- Denormalization: Trade write complexity for read performance
The Three-Layer Scaling Model
Practice Exercises
-
Estimation: A system currently handles 500 QPS with a single server at 70% CPU. If traffic grows to 2000 QPS, how many servers are needed? What if Amdahl's law applies with 5% sequential overhead?
-
Design: Design a horizontally scalable web application architecture. Consider: how do you handle session state? How do you handle file uploads? How do you deploy new versions without downtime?
-
Trade-offs: Compare vertical and horizontal scaling for a relational database. When is each approach appropriate? What are the cost implications at 10x, 100x, and 1000x scale?
-
Analysis: Draw a decision tree for choosing a load balancing algorithm based on: server heterogeneity, connection duration variability, and session persistence requirements.
What to Learn Next
-> Networking Fundamentals TCP/IP, HTTP, DNS, CDNs, and network latency.
-> API Design REST, GraphQL, gRPC, versioning, and rate limiting.
-> Databases SQL vs NoSQL, indexing, replication, and sharding.
-> Caching Strategies Redis, Memcached, cache invalidation, and write strategies.
-> Load Balancing Algorithms, health checks, and L4 vs L7.
-> CAP Theorem Consistency models, availability, and partition tolerance.