What model collapse is
Model collapse is the progressive degradation that happens when generative AI models are trained on data produced by other AI models instead of original human-created data. Picture a photocopier copying a copy of a copy: each pass loses a little detail, and after enough rounds the image is unrecognisable. Models behave similarly. When one generation learns from the outputs of the previous generation — and that from the one before — the system drifts away from the real-world distribution it was meant to capture, losing diversity and fidelity until its output becomes bland, repetitive, or incoherent. It is a distinctively modern risk, made plausible by the fact that the internet, the main training source, is filling with AI-generated text and images.
Why it happens
The mechanism is statistical, not mysterious. Any trained model is an imperfect approximation of the distribution it learned from: it slightly over-represents common patterns and under-represents rare ones, the “tails” of the data. When you then train a new model on the first model’s output, those small biases become the new ground truth, and the next model approximates that — compounding the distortion. Two effects dominate. First, the tails disappear: unusual but valid examples get sampled less and eventually vanish, so the model forgets edge cases and minority patterns. Second, variance collapses: outputs cluster ever more tightly around the average, draining the diversity that made the original data rich. After several recursive generations, the model can converge on a narrow, sometimes nonsensical slice of its original range.
The research evidence
The phenomenon was characterised in published studies that trained models recursively on their predecessors’ outputs and measured quality across generations. The consistent finding is that performance and diversity decline generation over generation, with the rare tail of the distribution eroding first. Crucially, the experiments show this happens even when each individual training step looks reasonable — the damage is cumulative and only obvious across multiple rounds. These are controlled demonstrations rather than evidence that today’s deployed frontier models have collapsed; their value is in proving the mechanism is real and in clarifying the conditions under which it bites.
Why it matters and how it is mitigated
The practical worry is data pollution: as more web content is AI-generated, naively scraping the internet for training data risks unknowingly training on machine output, inviting collapse over time. That makes genuine human data more valuable, not less — a strategic shift for the whole field. The defences are sensible rather than exotic: maintain a strong supply of fresh human data, filter or detect synthetic content before training, track data provenance so you know what you are learning from, and use synthetic data deliberately and in moderation (it can be useful when curated) instead of indiscriminately. Labs treat this as a managed risk, not a doomsday certainty. Model collapse is best understood as a real constraint that shapes how responsibly future models must be trained — a reason to value authentic human data and careful curation, not a prophecy that AI will inevitably degrade itself.