What does temperature actually do?

Temperature scales the model's output probabilities before sampling. Low values (near 0) make the model pick the most likely token almost every time — predictable and repetitive. High values (toward 1.5–2) flatten the distribution so less likely tokens get picked, giving variety and creativity at the cost of coherence.

Should I change temperature or top-p?

Change one, not both. Temperature reshapes the whole distribution; top-p (nucleus sampling) keeps only the smallest set of tokens whose probabilities sum to p, then samples among those. Most teams pick a temperature and leave top-p at 1.0, or vice versa.

Why is temperature 0 recommended for code and extraction?

For code, structured output, and data extraction you want the single most likely, correct continuation every time — not creative variety. Near-zero temperature maximises determinism and reproducibility, which matters for tests and pipelines.

Does a higher max-tokens cost more?

Max-tokens only caps how long the response can be; you pay for tokens actually generated. Setting it generously is fine, but a tight cap protects you from runaway responses and surprise bills on completion-heavy tasks.

LLM Temperature & Parameter Guide

LLM temperature & parameter guide

Sampling parameters quietly decide whether your LLM feels reliable or unhinged. Set temperature too high on a coding task and you get plausible-looking nonsense; set it too low on a brainstorm and you get the same three ideas every time. This guide maps the common task types to sensible starting parameters and explains the reasoning so you can adjust with intent instead of guessing.

How it works

Pick the task type and how deterministic you need the output to be. The tool returns a recommended temperature, top-p, and max-tokens, plus a short rationale. The logic follows the well-established trade-off: deterministic, correctness-critical tasks (code, extraction, classification) sit near temperature 0, balanced tasks (Q&A, summarisation) sit in the 0.2–0.5 band, and generative, divergent tasks (storytelling, brainstorming, marketing copy) climb toward 0.8–1.2. The determinism slider nudges the recommendation within the band so you can lean more consistent or more varied.

Tips and notes

Change temperature or top-p, not both — combining them makes results hard to reason about. For anything you will run repeatedly (tests, pipelines, eval suites) pin temperature to 0 so results are reproducible. If outputs feel repetitive at moderate temperature, a small bump plus a frequency_penalty often helps more than a large temperature jump. And remember that the “right” setting is the one that passes your own evaluation on real inputs — treat these as informed starting points, then measure.