What is exponential backoff?

Exponential backoff increases the wait between retries multiplicatively — for example 1s, 2s, 4s, 8s. This gives an overloaded or rate-limited service time to recover instead of hammering it with immediate retries, which is why it is the standard pattern for resilient API clients.

If many clients retry on the same fixed schedule they all hit the server at the same moments, creating synchronized spikes — the thundering-herd problem. Jitter randomizes each delay so retries spread out. Full jitter picks a random value between zero and the computed delay; equal jitter keeps half the delay fixed and randomizes the other half.

How do I pick a timeout budget?

Your overall request timeout should be at least the worst-case cumulative delay plus the time of each actual API call. The calculator gives you the cumulative backoff total so you can add your per-call latency on top and set a sensible deadline.

Should I retry every error?

No. Retry transient errors — HTTP 429 rate limits, 500/502/503/504, and network timeouts — but not 400-class client errors like a malformed request or an invalid API key, which will fail identically every time. Retrying those just wastes the budget and delays surfacing the real problem.

LLM API Retry Strategy Calculator

Size your retry behavior before it bites you in production

LLM APIs throttle and occasionally fail, so any serious integration needs retry logic. Get the parameters wrong and you either give up too soon or stack delays that blow past your request timeout. This calculator turns your base delay, retry count, multiplier, and jitter choice into a concrete schedule — every attempt’s delay, the running total, and the worst-case budget you need to allow.

How it works

For attempt number n (starting at zero), the base delay is multiplied by the multiplier raised to the power of n, then capped at the maximum delay you set. The tool then applies your chosen jitter mode: none leaves the delay exact; full jitter shows the range from zero to the computed delay; equal jitter shows half the delay fixed plus a random half. It sums the upper bound of every delay to give the worst-case cumulative wait, which is the figure you add your per-call latency to when choosing an overall timeout. The schedule is laid out attempt by attempt so you can see exactly how long a fully-failing request would take.

Tips and notes

A multiplier of 2 with full jitter is a well-tested default for most API clients. Set a maximum delay cap so a high retry count does not produce minute-long waits. Only retry transient failures — 429 and 5xx responses and network errors — and pass through 4xx client errors immediately. Remember the worst case stacks both the backoff and the actual call time on every attempt, so your timeout budget must cover both. When a provider returns a Retry-After header, honor it instead of your computed delay; the calculator is for the cases where it does not.