What does temperature do?

Temperature scales how sharply the model favours its most likely next token. Near 0 it almost always picks the top choice (focused, repeatable); higher values flatten the distribution so less likely tokens get picked more often (diverse, creative, riskier).

What is top-p (nucleus sampling)?

Top-p keeps only the smallest set of tokens whose probabilities add up to p, then samples from that set. A top-p of 0.9 ignores the long tail of unlikely tokens; lowering it makes output safer and more focused.

Should I change both temperature and top-p?

Usually pick one to tune and leave the other at its default. Adjusting both at once makes the combined effect hard to reason about. For most tasks, tuning temperature alone is enough.

What is a good temperature for factual answers?

For factual QA, extraction, or anything needing consistency, use a low temperature (0 to 0.3). For brainstorming or creative writing, 0.7 to 1.0 produces more varied output.

Temperature & Top-p Explainer

Pick sampling settings with confidence

Temperature and top-p control how random an LLM’s output is, but the numbers are abstract until you see what they do. Drag the sliders to read a plain-English explanation of each setting in real time, or load a task preset and copy the recommended values straight into your API call.

How sampling works

After processing your prompt, the model produces a probability for every possible next token. Sampling parameters decide how that distribution is turned into an actual choice:

Temperature rescales the probabilities. At 0 the model is nearly deterministic — it picks the most likely token almost every time. As temperature rises toward 1 and beyond, the distribution flattens, so unlikely tokens get chosen more often. High temperature means more variety and more risk of going off-topic.
Top-p (nucleus sampling) caps which tokens are even eligible. It keeps the smallest group of top tokens whose probabilities sum to p, then samples among those. A top_p of 0.9 trims the improbable tail; lowering it tightens the output.

The two interact, so the usual advice is to tune one and leave the other at its default.

Task-based starting points

Factual QA / extraction / classification — low temperature (0–0.3) for consistent, repeatable answers.
Code generation — low-to-moderate (0.2) so it stays correct but not rigid.
Summarization — moderate (0.3–0.5) to stay faithful with light rephrasing.
Creative writing / brainstorming — high (0.8–1.0) for variety and surprise.

When in doubt, start at the preset, generate a few samples, and nudge upward only if the output feels too repetitive.