What pruning does
Model pruning removes the parts of a trained neural network that matter least, leaving a smaller model that does almost the same job. Large networks are typically over-parameterised: they contain far more weights than any single task needs, and many of those weights end up near zero or barely affect the output. Pruning identifies and deletes this redundancy, producing a model that uses less memory, runs faster, and is cheaper to deploy — without retraining from scratch.
Magnitude pruning
The simplest and most common approach is magnitude pruning: rank weights by their absolute value and remove the smallest ones, on the assumption that tiny weights contribute little to the output. You choose a sparsity target — say, removing 50% of weights — set those below the threshold to zero, and then fine-tune the survivors. Despite its simplicity, magnitude pruning is a strong baseline, and modern variants prune gradually during training or score weights by more sophisticated importance measures than raw magnitude.
Structured versus unstructured pruning
Pruning comes in two flavours that differ in what gets removed. Unstructured pruning zeroes out individual weights anywhere in the network. This can reach high sparsity with minimal accuracy loss, but the resulting irregular sparsity is hard for standard hardware to exploit, so the model may not actually run faster without specialised support. Structured pruning removes whole components — neurons, convolutional channels, attention heads, or entire layers — so the model stays dense and genuinely shrinks, delivering real speedups on ordinary GPUs and CPUs at the cost of being a coarser, less precise cut.
The lottery ticket hypothesis
Pruning research produced a striking idea: the lottery ticket hypothesis. It claims that inside a large, randomly initialised network there is a small subnetwork — a “winning ticket” — that could have been trained on its own, from the same initial weights, to match the full network’s accuracy. If true, it implies that dense training is partly a search for these sparse high-performing subnetworks, and that very small models with the right structure are capable of the same results as much larger ones. It reframed pruning from a mere compression trick into a window on how networks learn.
Practical pruning workflows for LLMs
In practice, teams rarely prune once and stop. The standard recipe is iterative prune-and-fine-tune: remove a fraction of the least important structure, fine-tune to recover accuracy, then repeat until the size or speed target is met. For large language models, structured pruning of attention heads, feed-forward dimensions, or whole layers is favoured because it produces models that actually run faster on real hardware, and it pairs naturally with quantization and distillation. Each round is evaluated against the deployment budget, balancing how much to remove against how much accuracy can be tolerated.