Retries need a budget, too

A request times out. Retrying sounds reasonable. But a timeout tells you that the caller stopped waiting, not necessarily that the server stopped working.

That distinction matters when the operation creates something. A second attempt could create a second order, job, or message. Before adding retry logic, decide how the system recognizes repeated attempts at the same intended operation.

A timeout is an observation

The server may have completed the operation while the response was delayed or lost. The caller’s local view is incomplete. Retrying a read is often simpler than retrying a write, but both still consume resources.

An idempotency mechanism can associate multiple attempts with one logical operation. Its scope, retention period, and behavior for conflicting inputs need to be explicit. A random key isn’t useful if the client generates a new one for every retry.

Leave room for recovery

Immediate retries add traffic at the moment a dependency may already be struggling. Waiting between attempts reduces pressure. Increasing that wait across attempts gives a service more room to recover; randomizing it can reduce synchronized bursts from many clients.

Amazon’s guide to timeouts, retries, and backoff with jitter (opens in a new tab) explains how retries can amplify load and why backoff, jitter, and idempotency belong in the same design discussion.

Count the whole call chain

If several layers each retry independently, one user request can produce many downstream attempts. Give retry ownership to a deliberate layer and make the total budget visible.

The budget includes time as well as attempts. A caller with a two-second deadline gains nothing from a retry scheduled after that deadline. Pass useful cancellation signals through the work where the stack supports them.

Know when to stop

Some failures are temporary. Others require a different input, permission, or configuration. Repeating an unchanged request won’t fix every error.

Classify the failures your application actually encounters. Cap attempts, respect an overall deadline, record the last outcome, and return enough information for the caller to decide what to do next. A clear failure is much easier to deal with than a retry loop nobody can see.