Tokens per minute is the throughput cap most LLM providers enforce. If the tokens you send in a rolling minute exceed it, requests get 429 rate-limit errors and must be retried with backoff.

How is drain time calculated?

The planner compares the burst's token demand against your TPM capacity. If demand exceeds capacity, the backlog drains at the rate of spare capacity per second, and the tool reports how many seconds that takes.

Why does retrying cost more?

Failed requests that get retried still consume some overhead, and aggressive backoff can mean re-sending prompt tokens. The retry cost multiplier lets you model that extra spend versus a clean queued approach.

Does this replace a real load test?

No. It is a fast planning estimate to size queues and limits before you build. Always validate against your provider's actual behaviour and your real token distribution.

Rate Limit Burst Capacity Planner

Rate limit burst capacity planner

Traffic rarely arrives evenly. A scheduled job, a viral moment, or a batch import can dump thousands of requests into a few seconds — and if that exceeds your provider’s tokens-per-minute (TPM) limit, you get a wall of 429 errors. This planner models the burst, tells you whether it fits, and sizes the queue depth and drain time you need to absorb it gracefully.

How it works

The planner converts your burst into a token demand and compares it to your TPM capacity expressed per second:

burst_tokens   = requests × avg_tokens_per_request
capacity/sec   = TPM / 60
demand/sec     = burst_tokens / burst_duration

If demand per second exceeds capacity per second, a backlog forms. The peak queue depth is the overflow, and it drains at the rate of spare capacity once the burst ends. The tool also applies your retry cost multiplier to estimate the extra spend a naive retry-on-429 strategy would incur versus smooth queuing.

Tips and notes

Size your queue for the peak burst, not the average — averages hide the spikes that actually trip the limit.
Prefer a token-bucket queue with jittered exponential backoff over blind retries; it keeps you just under the limit and avoids retry storms.
If drain time is unacceptably long, the real fix is a higher TPM tier or splitting traffic across multiple keys or providers, not more retries.