What is chat template overhead?

Before a model sees your messages, the chat template wraps each turn in special tokens — a role header, turn-start and turn-end markers, and often a sequence BOS/EOS. These tokens are invisible in your message text but are still counted and billed, so a conversation of short messages can carry surprising overhead.

Why does overhead matter more for short messages?

The template adds a roughly fixed number of tokens per turn regardless of message length. For a 500-token answer, a 7-token wrapper is noise. For a 5-token "yes/no" exchange in a high-volume classifier, the wrapper can be larger than the content itself, sometimes doubling the bill.

How much overhead do typical templates add?

It varies by family. ChatML adds a handful of tokens per turn (role header plus start/end markers). Llama 3 uses header-id tokens plus an end-of-turn token per message and a sequence BOS. The analyzer estimates each so you can compare; exact counts depend on the tokenizer.

Can I reduce template overhead?

You cannot remove the template on a chat endpoint, but you can reduce turn count by batching, trim system-prompt repetition, and prefer providers that cache the system prompt so its tokens (and template) are not re-billed on every call. For very high-volume short calls, a completion (non-chat) endpoint avoids per-turn wrappers entirely.

Is anything uploaded?

No. Token estimation and template math run locally in your browser. Nothing you enter is sent to a server.

Chat Template Token Overhead Analyzer

Chat template token overhead analyzer

Every chat API silently wraps your messages in template tokens — role headers, turn delimiters, sequence markers — and bills you for all of them. For long answers this is rounding error; for a high-volume classifier sending five-token messages, the wrapper can cost more than the content. This tool estimates that hidden overhead across four model families so you can see it in numbers.

How it works

The analyzer counts the tokens in your visible message content, then adds each model family’s per-turn template tokens:

ChatML (OpenAI): a turn-start marker, a role header, and a turn-end marker per message.
Llama 3: a begin-of-text token once, plus start/end header-id tokens and an end-of-turn token for each message.
Mistral: instruction-wrapper tokens around user turns plus sequence BOS/EOS.
Gemma / Gemini: start-of-turn and end-of-turn markers per message.

For each family it reports total template tokens, total prompt tokens, and the overhead percentage = template / (content + template). Content tokens are estimated from character count at roughly four characters per token.

Tips and notes

Overhead is dominated by turn count, not message length — fewer, larger turns are cheaper per token than many tiny ones.
A repeated system prompt pays its content and template cost on every call unless your provider caches it; prompt caching is the single biggest lever for chat-heavy workloads.
For extreme-volume short calls (moderation, classification), consider a non-chat completion endpoint to skip per-turn wrappers entirely.
The percentages here are estimates; trust the API’s returned usage for billing, and use this tool to decide whether the overhead is worth optimizing.