Modern System Design & API Architecture · Chapter 2 · ~15 min · Beginner
Scaling Up: Vertical vs Horizontal & Load Balancing
Learn how systems handle millions of simultaneous users without crashing: the Single Point of Failure, Vertical Scaling vs Horizontal Scaling, and how Load Balancers distribute traffic.
- Identify Single Points of Failure (SPOF) in early-stage architectures
- Understand the tradeoffs of Vertical Scaling (Scale Up)
- Master Horizontal Scaling (Scale Out) used by tech giants
- Distribute web traffic with Round Robin Load Balancers
1The Scaling Problem: What Happens When You Go Viral?
Imagine you build a website for your startup and host it on one computer. When you have 50 friends testing it, the server is fast and responsive.
Then your product gets featured on the news or goes viral on TikTok. Suddenly, 100,000 visitors arrive at the exact same minute.
Your single server's CPU hits 100%, RAM fills up completely, and incoming connection requests begin queuing up. The server crashes with a dreaded 500 Internal Server Error.
This is called a Single Point of Failure (SPOF): a single component whose failure brings down the entire system.
What is a 'Single Point of Failure' (SPOF) in system design?
2Vertical Scaling (Scale Up): Buying a Bigger Box
When your server is running out of horsepower, the easiest initial fix is Vertical Scaling: upgrade the machine to more CPU cores, faster clocks, and 512 GB of RAM.
Dead Simple
Zero code changes required. Your app stays identical, running on one big computer.
Hard Physical Ceiling
You hit physical hardware limits. Monster machines cost exponentially more money, and it is still a Single Point of Failure!
What is the primary limitation of relying solely on Vertical Scaling?
3Horizontal Scaling (Scale Out): The Power of Numbers
Instead of buying one giant supercomputer, Horizontal Scaling means adding dozens or hundreds of smaller, inexpensive commodity servers side by side.
If 1 server can handle 1,000 users/second, 10 servers can handle 10,000 users/second. Need to handle 50,000? Just spin up 50 servers!
Why do large companies like Netflix prefer Horizontal Scaling over Vertical Scaling?
4The Load Balancer: The Traffic Cop
When you have 5 backend servers running, how does a user's phone know which server to send its request to?
You place a Load Balancer in front of your server pool. The Load Balancer acts like a traffic cop:
1. Single Public IP Address
Users connect directly to the Load Balancer's public IP address, keeping your actual application servers private and safe from direct attacks.
2. Round-Robin Distribution
The load balancer sends Request 1 to Server A, Request 2 to Server B, Request 3 to Server C, repeating in a circle so no server gets overwhelmed.
3. Health Checks
The load balancer pings each server every 5 seconds (GET /healthz). If Server B stops responding, the load balancer automatically stops sending traffic to Server B until it recovers!
What does a Load Balancer do when one of the servers in its pool fails a health check?
5Hands-On: Test Traffic Scaling Live
Use the interactive simulator below. Drag the traffic slider to see when a single server fails, then toggle the Load Balancer on to balance the load!
Live Traffic Scaling & Load Balancer Simulator
Notice: At low traffic, 1 server is fine. Drag traffic past 1,500 req/s to see it struggle.
6Chapter Knowledge Challenge
Test your mastery of scaling strategies, Single Points of Failure, and load balancer algorithms:
What is the critical danger of relying solely on Vertical Scaling (Scale Up) for a production system?
How does Round Robin load balancing distribute traffic across 3 servers?
7Recap
- A Single Point of Failure (SPOF) is any individual component whose failure takes down the whole system.
- Vertical Scaling (Scale Up) adds CPU/RAM to a single machine, but hits cost and physical limits.
- Horizontal Scaling (Scale Out) adds multiple machines in parallel for infinite growth and redundancy.
- A Load Balancer distributes incoming user traffic across servers and routes around failed machines via health checks.
Congratulations on completing Chapter 2! Mark as complete to update your syllabus: