Two levers, one goal: fit the device budget
Running AI on a phone, a smart camera, or a microcontroller means fitting a model into a fraction of a cloud server's memory and compute budget. Quantization reduces the numeric precision used for weights and activations — typically from 32-bit floats down to 16-bit, 8-bit, or even 4-bit integers — shrinking model size and speeding up inference roughly in proportion. Pruning removes weights or whole neurons/channels that contribute little to the output, shrinking size further.
Why the trade-off bends instead of sloping evenly
Compression rarely trades accuracy for size in a straight line. Moderate quantization and moderate pruning are close to free, because the model was carrying redundant precision and redundant weights to begin with. Push either lever hard enough and the curve bends sharply: past some threshold you are no longer removing redundancy, you are removing capacity the model genuinely needs, and accuracy collapses much faster than size continues to shrink.
Three device tiers, three budgets
A cloud GPU has essentially no meaningful size constraint. A phone has an app-size budget, often tens of megabytes. A microcontroller running tinyML has a flash budget frequently measured in single-digit megabytes or even kilobytes — the tightest tier by a wide margin, and the one where every bit of quantization and pruning matters most.
What this trades away, beyond accuracy
Edge deployment isn't purely a downgrade: it trades some accuracy and capability for lower latency, privacy (data never leaves the device), reliability without a network connection, and lower cost at scale. Whether that trade is worth it depends entirely on the application — a wake-word detector running on a battery for a year cares about none of the things a cloud recommendation model optimizes for.
🧪 Try it yourself: the Edge AI Lab simulation lets you experiment with everything described above directly in your browser.