What a context window is
A model’s context window is the maximum amount of text — measured in tokens — it can consider at once, including your prompt, any documents you paste, and its own reply. Around 750 words equal roughly 1,000 tokens, so a 200K window holds about 150,000 words: a few long reports or a small book. A larger window means you can feed in more material without chunking, but it does not by itself guarantee the model will reason over all of it well.
Window sizes across the major models
- Claude 3 (Anthropic): 200K tokens as standard, with larger windows for some enterprise tiers.
- GPT-4 Turbo (OpenAI): 128K tokens.
- Gemini 1.5 Pro (Google): up to 1 million tokens generally available, with a 2M-token preview — the largest published window.
- Llama 3 (Meta, open source): smaller base windows (8K), extended to 128K in later releases; varies by checkpoint and host.
On raw size, Gemini leads by a wide margin, while Claude and GPT-4 Turbo occupy the comparable 128K–200K band that covers most practical long-document tasks.
Why size is not the whole story
Bigger windows expose a quality problem: recall does not stay uniform across a long context. Research on the lost-in-the-middle effect shows models retrieve facts near the start and end of their input more reliably than facts in the middle. So a model with a 1M-token window may still miss a detail buried at the 400K mark. The meaningful metric is effective context — how much of the window the model can actually use accurately — which is usually smaller than the advertised maximum and varies by model.
Real-world performance
In practice, Claude is well regarded for sustained reasoning over long documents and tends to hold coherence across its 200K window. Gemini’s massive window is genuinely useful for tasks like analysing entire codebases or long video transcripts, though performance can soften deep in the context. GPT-4 Turbo’s 128K is ample for most reports and conversations. For all of them, accuracy degrades and latency and cost rise as you fill the window, so “use the biggest window available” is rarely the optimal strategy.
Long context vs retrieval
Two approaches solve “the model needs to know about my data”: stuff it into a long context, or use retrieval-augmented generation (RAG) to fetch only the relevant chunks. Long context is simplest for a single document that fits and benefits from whole-document reasoning. RAG wins when the corpus is large, frequently updated, or cost-sensitive — you pay to process only the retrieved snippets, not the entire library, on every query. Many production systems combine both: retrieve the relevant sections, then give the model a generous window to reason over them. Choose based on corpus size, freshness, and budget rather than on which model advertises the largest number.