Scale out behind a load balancer
Compare vertical and horizontal scaling and spread traffic across servers.
- Contrast vertical and horizontal scaling.
- Explain why stateless services are easy to scale out.
- Choose between common load-balancing algorithms.
Vertical scaling means a bigger machine: simple, but it hits a hardware ceiling and leaves a single point of failure. Horizontal scaling means more machines doing the same job, which adds capacity and redundancy - but only if any server can handle any request. That is why well-designed application tiers are stateless: sessions and data live in shared stores such as a database or Redis, not in server memory.
How load balancers spread traffic
- Round robin: send each request to the next server in turn. Simple and fair when requests cost about the same.
- Weighted round robin: bigger servers get proportionally more requests.
- Least connections: send to the server with the fewest active connections - good when request durations vary.
- Hashing (by client IP, user id, or with consistent hashing): the same key goes to the same server, useful for cache locality.
A layer 4 balancer routes on IP and port. A layer 7 balancer understands HTTP, so it can route by path or header (for example /api to one pool, /images to another) and terminate TLS. Both use health checks to stop sending traffic to failed instances.
The load balancer must not become the new single point of failure. Production setups run redundant balancers (an active-passive pair with a floating IP, or several behind DNS or anycast). Autoscaling adds and removes application servers as load changes, which only works smoothly because those servers are stateless.
Practice: choose how to balance
Try it
Up to 15 XPMatch the algorithm to the traffic
In an interview you’ll be asked why you picked one. Find what survives each pattern, and work out what the others get wrong.
app-c has four times the capacity of the others, and every request costs the same.
- app-abusiest: 4 of 3, handled 80
- app-bbusiest: 4 of 3, handled 80
- app-cbusiest: 4 of 12, handled 80
The bar is the busiest moment. The line marks what the backend can handle.
0 of 2 scenarios solved
Try it
Up to 15 XPCan you add a server to this?
Horizontal scaling only works if any server can handle any request. Decide which of these designs allow that.
“Sessions live in Redis.”
“Uploaded files are saved to the server’s local disk.”
“A signed token carries the user’s identity.”
“A user’s cart is held in server memory.”
“Each server runs its own nightly cleanup job.”
Key takeaways
Scale out stateless services; keep state in shared stores.
Pick round robin for uniform requests, least connections for uneven ones, hashing for locality.
Health checks and redundant balancers keep the tier available.
Lesson quiz
6 questions · pass with 5 correct · up to 50 XP
Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.
Questions about this lesson
Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.
Loading posts…