Use health checks and connection draining
Remove failing instances carefully and let in-flight work finish.
- Distinguish liveness from readiness and explain draining and health-check hysteresis.
A liveness check asks whether a process should be restarted; a readiness check asks whether it should receive traffic. A shallow check can pass while a critical dependency is unavailable, while an overly deep check can eject every backend during a shared dependency outage. Thresholds, separate failure domains, and recovery hysteresis reduce flapping. During deploys, draining stops new connections while giving in-flight requests time to finish.
Real draining behavior depends on protocol and client behavior. Long-lived WebSockets or streaming responses may require a longer grace period or explicit reconnect logic.
Practice: tune a health check
Try it
Up to 15 XPFind a threshold you can live with
The top row is real check results: a blip, a stretch of flapping, then a real outage. The bottom row shows what the balancer decides.
Start at 1 and 1, then raise them. Which number gets better, and which gets worse? No setting gets all three to zero.
- Changes of state
- 10
- Failing but still getting traffic
- 0
- Working but removed
- 0
Practice: drain a backend for a deploy
Try it
Up to 15 XPRemove a backend without dropping requests
Step through a deploy and predict what the balancer does at each stage.
Try it
Up to 15 XPLiveness or readiness?
Liveness asks whether to restart it. Readiness asks whether to send it traffic. Decide which check should react to each situation.
“The process is deadlocked and will never recover by itself.”
“The app is still warming its cache after starting.”
“The instance is temporarily overloaded.”
“The instance is shutting down for a deploy.”
“A memory leak has left the process unresponsive.”
Key takeaways
Liveness and readiness serve different purposes.
Avoid health checks that turn one dependency failure into total pool ejection.
Drain connections before planned removal when the protocol allows it.
Lesson quiz
5 questions · pass with 4 correct · up to 50 XP
Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.
Questions about this lesson
Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.
Loading posts…