How many characters are in a token?

For English, roughly 4 characters per token. But this varies wildly by content — Chinese can be close to 1 character per token, and emoji can be 2 or more tokens each.

Why is non-English text so much more expensive?

LLM tokenizers are trained mostly on English, so other scripts fragment into many small tokens. Chinese, Arabic, and emoji can cost several times more tokens per character than English prose.

Why include a code modifier?

Source code is dense with symbols and indentation that each become separate tokens, so it produces more tokens per character than ordinary prose.

It is a calibrated estimate, typically within 5-10% for English and somewhat looser for non-Latin scripts. For exact counts, use the model's own tokenizer.

Characters-to-Tokens Converter

Characters-to-tokens converter

Sometimes all you have is a character count — from a CMS field, a database column, or a text-length limit — and you need to know how many tokens that is for an LLM. This converter does the lookup for GPT-4o, Claude, Gemini, and Llama, with modifiers for code and non-Latin scripts that dramatically change the ratio.

How it works

For English prose, the rule of thumb is about 4 characters per token. But that number is only true for English: LLM tokenizers are trained predominantly on English text, so other scripts fragment into many small tokens. The tool applies a language/content modifier on top of each model’s base ratio — Chinese and Japanese can approach 1-2 characters per token, Arabic fragments heavily, emoji often cost 2+ tokens each, and code runs denser than prose.

Tips and notes

If your product serves global users, budget for the worst-case language, not English — a prompt that fits comfortably in English can overflow a context window once translated to Chinese or Arabic. For mixed documents, the estimate is a blend, so leave extra margin. To convert from word counts instead, use the words-to-tokens converter; for exact, billing-grade numbers, always confirm with the model’s own tokenizer.