Large neural networks are expensive to run: every inference means moving millions of weights through memory and multiplying them against activations. This lab visualizes three of the main techniques engineers use to shrink networks for cheaper, faster inference — pruning, quantization and knowledge distillation — applied live to a small feed-forward network.
Combining these techniques compounds their savings: an 8-bit quantized, 70%-pruned, distilled model can be over an order of magnitude smaller than its full-precision teacher while keeping most of its accuracy — the difference between a model that needs a data-center GPU and one that runs on a phone.
An interactive 3D neural network that you compress live — prune away weak connections by magnitude, quantize the survivors into a shrinking ladder of bit-widths, and switch on knowledge distillation to watch a compact student network absorb the teacher's behaviour.
Edges are globally sorted by weight magnitude, so raising the pruning slider strips the weakest connections first — exactly how real magnitude pruning works. Lowering the quantization bit-width rounds every surviving weight onto a coarser ladder of values, visible as color banding on the connections.
Pick a network size, then drag the pruning and quantization sliders and watch the model size, inference speed-up and estimated accuracy update live. Toggle distillation to reveal a small student network being fed synthetic "knowledge" particles from the teacher's output layer.
Production compression pipelines usually stack these techniques: a pruned, 8-bit quantized, distilled model can be 10-20× smaller than its full-precision teacher while retaining most of its accuracy, which is how large models end up running on phones and microcontrollers.