A neural network trained on a datacenter GPU cluster is often enormous relative to where it eventually needs to run: a phone, a browser tab, a microcontroller at the edge of a network. Model compression covers the techniques that shrink an already-trained network so it fits and runs quickly in that much smaller environment, without retraining from scratch.
Pruning: removing what does not matter much
Pruning identifies which weights in a trained network contribute least to its output, typically ranked by magnitude, and removes them entirely. Counterintuitively, a network often tolerates removing a large fraction of its smallest weights with very little accuracy loss, because much of a trained network's capacity turns out to be redundant.
Quantization: fewer bits per weight
Quantization keeps every weight but stores each one with less numeric precision: 16-bit or 8-bit instead of the 32-bit floating point typically used during training. Lower precision means a smaller file and faster arithmetic, at the cost of some rounding error in every weight.
Where the cliff is
Both techniques degrade accuracy gracefully at first, then fall off a cliff once pushed too far. A model can often lose 70-80% of its smallest weights, or drop to 8-bit precision, with negligible accuracy change. Push past that point, prune too aggressively or quantize too coarsely, and accuracy collapses quickly. Finding where that cliff sits for a specific model is largely empirical.
Try it yourself
The Model Compression Lab lets you dial pruning and quantization independently and watch model size, weight distribution, and test accuracy respond in real time.
🧪 Try it yourself: the Model Compression Lab simulation lets you experiment with everything described above directly in your browser.