Use timeouts and retries safely
Choose retry policies that recover from transient failures without multiplying work.
- Explain deadlines, bounded retries, backoff, jitter, and idempotency keys.
Set a deadline for each request so callers do not wait forever. Retry only failures that may be transient, cap attempts, and use exponential backoff with jitter so many clients do not retry together. A retry can repeat an operation whose first result was merely delayed. Idempotent operations have the same effect when repeated; an idempotency key can let a server recognize duplicate submissions.
Real clients randomize delays and respect an overall deadline. A retry policy should also avoid retrying permanent errors such as invalid input.
Practice: survive an outage with retries
Try it
Up to 15 XPWhat do your retries do to a recovering server?
The server goes down for ten ticks and drops every open connection. The red and green marks show when it fails and returns. Compare the policies:
- Retry immediately. How far above capacity does the load go, and how many requests give up? They burned every attempt while the server was still down.
- Exponential backoff. Nothing gives up, because the retries outlast the outage. But look at the spikes: clients that failed together retry together.
- Turn on jitter. What happens to the biggest spike?
- No retries has the calmest chart. What did it cost?
- Requests
- Server capacity
Show the numbers as a table
| Time (ticks) | Requests | Server capacity |
|---|---|---|
| 0 | 10 | 15 |
| 1 | 10 | 15 |
| 2 | 10 | 15 |
| 3 | 10 | 15 |
| 4 | 10 | 15 |
| 5 | 10 | 15 |
| 6 | 10 | 15 |
| 7 | 10 | 15 |
| 8 | 10 | 15 |
| 9 | 10 | 15 |
| 10 | 70 | 15 |
| 11 | 80 | 15 |
| 12 | 90 | 15 |
| 13 | 100 | 15 |
| 14 | 110 | 15 |
| 15 | 120 | 15 |
| 16 | 60 | 15 |
| 17 | 60 | 15 |
| 18 | 60 | 15 |
| 19 | 60 | 15 |
| 20 | 60 | 15 |
| 21 | 55 | 15 |
| 22 | 50 | 15 |
| 23 | 45 | 15 |
| 24 | 40 | 15 |
| 25 | 35 | 15 |
| 26 | 30 | 15 |
| 27 | 25 | 15 |
| 28 | 20 | 15 |
| 29 | 15 | 15 |
| 30 | 10 | 15 |
| 31 | 10 | 15 |
| 32 | 10 | 15 |
| 33 | 10 | 15 |
| 34 | 10 | 15 |
| 35 | 10 | 15 |
| 36 | 10 | 15 |
| 37 | 10 | 15 |
| 38 | 10 | 15 |
| 39 | 10 | 15 |
| 40 | 10 | 15 |
| 41 | 10 | 15 |
| 42 | 10 | 15 |
| 43 | 10 | 15 |
| 44 | 10 | 15 |
| 45 | 10 | 15 |
| 46 | 10 | 15 |
| 47 | 10 | 15 |
| 48 | 10 | 15 |
| 49 | 10 | 15 |
| 50 | 10 | 15 |
| 51 | 10 | 15 |
| 52 | 10 | 15 |
| 53 | 10 | 15 |
| 54 | 10 | 15 |
| 55 | 10 | 15 |
| 56 | 10 | 15 |
| 57 | 10 | 15 |
| 58 | 10 | 15 |
| 59 | 10 | 15 |
- Busiest moment
- 120 (capacity 15)
- Failed attempts
- 1035
- Requests given up on
- 110
- Overloaded after it came back
- 9 ticks
Practice: is this safe to repeat?
Try it
Up to 15 XPIdempotent or not?
Imagine the operation runs twice because a retry arrived. Would the result be different?
“Set the customer’s email to ana@example.com.”
“Add $10 to the account balance.”
“Read an order’s status.”
“Send a welcome email.”
“Delete order 123.”
“Create a new order.”
Try it
Up to 15 XPShould the client retry?
Retry only when a second attempt could turn out differently.
“503 Service Unavailable”
“400 Bad Request”
“Connection reset”
“403 Forbidden”
“429 Too Many Requests”
Key takeaways
Use an overall deadline and bounded attempts.
Backoff plus jitter reduces synchronized retry load.
Idempotency keys help safely repeat side-effecting requests.
Lesson quiz
5 questions · pass with 4 correct · up to 50 XP
Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.
Questions about this lesson
Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.
Loading posts…