What is time-to-first-token (TTFT)?

TTFT is the delay between sending a request and the first streamed token arriving. In streaming chat UIs it is what users perceive as responsiveness, far more than total generation time, so it is the key latency metric to optimise.

Where do the TTFT figures come from?

They are representative editable estimates based on published benchmarks and typical observed latencies. Real TTFT depends on region, load, prompt length and provider routing, so confirm against your own measurements.

Why does a faster model sometimes cost more?

Smaller, distilled models stream first tokens quickly and cost less, while large frontier models can be slower and pricier. But not always — the tool shows the actual trade-off so you do not assume cheap means slow.

Does output length affect TTFT?

TTFT itself is about the first token, so output length mostly affects total time and cost, not TTFT. The tool uses output length only to compute per-call cost, which it weighs against the latency requirement.

First-Token Latency vs Cost Optimizer

First-token latency vs cost optimizer

In a streaming chat UI, users judge speed by how fast the first token appears — time-to-first-token (TTFT) — not by total generation time. The fastest-streaming model isn’t always the cheapest, and the cheapest isn’t always fast enough. This optimizer ranks models by TTFT against cost per call so you can pick the cheapest model that still meets your latency bar.

How it works

You set a maximum acceptable TTFT and a per-call cost budget. The tool checks each model’s estimated TTFT and computes its cost for your output length, then filters to the models that satisfy both constraints:

qualifies = (model_ttft ≤ ttft_limit) AND (call_cost ≤ budget)
call_cost = (input + output tokens) priced at the model's rates

Among the qualifying models it highlights the cheapest, so you get the best price without breaking the responsiveness your UX needs.

Tips and notes

Optimise TTFT, not total latency, for chat — streaming hides total generation time as long as the first token lands fast.
A small “router” model can stream an instant acknowledgement while a larger model works in the background, decoupling perceived latency from quality.
TTFT varies with region and load; pin your provider region close to users and re-measure under real traffic before locking a model in.