Temperature is the single most useful knob for controlling how a language model behaves. In one number it decides whether your output is focused and predictable or varied and creative. Understanding it lets you stop fighting the model and start dialling in exactly the behaviour each task needs.
What temperature is doing under the hood
When a language model generates text, it does not pick one definite next word. It produces a probability distribution over all possible next tokens — “the” at 40%, “a” at 15%, and so on. Temperature reshapes that distribution before the model samples from it. A low temperature sharpens the distribution so the highest-probability token dominates, making the model almost always choose the “safe” continuation. A high temperature flattens the distribution, giving less likely tokens a meaningful chance of being picked. So temperature is not making the model smarter or dumber — it is changing how boldly it gambles on its own less-confident options.
What the values look like in practice
The range is typically 0.0 to 2.0, with 1.0 as the unmodified baseline.
- 0.0 — Near-deterministic. The model takes the most probable token almost every time. Ask it to list the planets and you get the same clean answer each run. Ideal for facts, code, and extraction.
- 0.5 — Lightly varied but still grounded. Good for summaries and explanations where you want natural phrasing without surprises.
- 1.0 — The default balance of coherence and variety. A reasonable starting point for general writing.
- 2.0 — Highly random. The model frequently picks unlikely tokens, which produces novel and sometimes surprising text — but also a much higher rate of rambling, off-topic, or nonsensical output.
Choosing the right temperature
Match the value to the cost of being wrong. For tasks where there is essentially one correct answer — running code, extracting fields, classifying text, answering factual questions — keep temperature low (0 to 0.3) so the model does not wander into a creative but incorrect continuation. For tasks where variety is the point — naming ideas, drafting marketing copy, fiction, brainstorming — raise it toward 0.7 to 1.0 to unlock range. Reserve values well above 1.0 for deliberate idea generation where you expect to sift through and discard a lot.
A common pitfall: changing too many knobs
Temperature is often confused or combined with top_p, another sampling control.
Both shape randomness, and most providers advise adjusting one or the other, not
both at once, because their effects interact in ways that are hard to reason
about. The practical habit is simple: leave top_p at its default and tune
temperature alone. Start at the value your task suggests, generate a few
samples, and nudge up if the output feels too repetitive or down if it feels too
unstable. That single dial, used deliberately, covers the large majority of
real-world control you will ever need.