Home▸Data Science▸PackNet Continual Learning: Weight Packing by Pruning, Shrinking Capacity and Zero Forgetting

PackNet Continual Learning: Weight Packing by Pruning, Shrinking Capacity and Zero Forgetting

Learn a sequence of tasks in one network by keeping only the largest weights of each task and freezing them, as PackNet does. See the free-weight budget shrink geometrically, the later tasks starve, and compare per-task error with plain fine-tuning that forgets. Checked against the closed-form capacity, a brute-force magnitude sort and the forgetting recursion. A Blender-built data bench with a weight-budget grid.

Data Science3DAdvanced60 FPS📱 Mobile-adapted🟠 Blender⇄ 2D version
PackNet Continual Learning: Weight Packing by Pruning, Shrinking Capacity and Zero Forgetting ↗ Open standalone

PackNet solves catastrophic forgetting by structural guarantee rather than by hoping regularization holds: it iteratively prunes a fixed-size network, freezing the most important surviving weights into a permanent binary mask for each task and handing the rest of the capacity forward to the next one. This simulator renders that process as a 16×16 grid of neural weights in 3D — each "Train Next Task" pass trains the free slots, ranks them by simulated importance, and freezes the top fraction (set by the Keep ratio slider) into a new colored, permanently frozen layer. Because frozen weights are never touched again, each task's accuracy — modeled as a saturating function of how many weights it was allocated relative to its difficulty — is locked in forever once its mask is cut, and the live per-task accuracy list proves it: earlier tasks never regress as later ones are added, right up until the shrinking free pool runs out.

⚙ Under the hood

Learn a sequence of tasks in one network by keeping only the largest weights of each task and freezing them, as PackNet does. See the free-weight budget shrink geometrically, the later tasks starve, and compare per-task error with plain fine-tuning that forgets. Checked against the closed-form capacity, a brute-force magnitude sort and the forgetting recursion. A Blender-built data bench with a weight-budget grid.

continual-learningpacknetpruningneural-networkscatastrophic-forgettingmulti-taskPackNetContinual LearningCatastrophic ForgettingPruningDeep Learning

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)