Definition
In AI, generation is the process by which a trained model produces new content — text, images, audio, or video — that did not exist in its training set. For a large language model, generation means predicting one token at a time and feeding each prediction back in to produce the next, until a stopping condition is reached. Generation is the defining behaviour of generative models, as opposed to models that only classify or score existing inputs.
Generative vs discriminative models
The clearest way to understand generation is by contrast. A discriminative model learns the boundary between categories — given an input, it outputs a label, such as “spam” or “not spam”. A generative model instead learns the distribution of the data well enough to sample from it, producing entirely new examples. Generation requires modelling far richer structure, which is why generative models such as LLMs and diffusion models are so much larger and more capable than simple classifiers.
Sampling strategies
When a language model generates text it produces a probability distribution over the next token, and a sampling strategy decides which token to actually pick:
- Greedy decoding — always take the single most probable token. Fast and deterministic, but tends to be repetitive and bland.
- Beam search — track several high-probability candidate sequences in parallel and keep the best overall. Common in translation, less so in open chat.
- Top-k sampling — sample randomly from only the k most likely tokens, injecting controlled diversity.
- Nucleus (top-p) sampling — sample from the smallest set of tokens whose cumulative probability exceeds p, adapting the candidate pool to how confident the model is.
Temperature and creativity
Sampling is usually combined with a temperature parameter that reshapes the distribution before a token is chosen. Lower temperatures concentrate probability on the most likely tokens for focused, predictable output; higher temperatures spread it out for more varied, creative — but potentially less coherent — output. Tuning temperature alongside top-k or top-p gives fine control over the trade-off between reliability and originality.
Why it matters
Generation is what makes modern AI feel transformative: it is the mechanism behind chatbots writing essays, models painting images from prompts, and tools composing music. Understanding that generation is fundamentally about sampling from a learned distribution explains both its creativity and its limitations — including why the same prompt can yield different outputs and why generated content can be confidently wrong.