LoRA Adapter Lab: Low-Rank Fine-Tuning Without Forgetting

This simulator renders a real weight matrix as a 3D heightfield and shows exactly how Low-Rank Adaptation (LoRA) fine-tunes it: instead of training every entry of a d×d matrix, two small factor matrices B and A are trained so that their product, scaled by α/r, forms an additive update on top of a frozen base. Switch between two independent task adapters — each with its own target pattern and its own B/A pair — and watch gradient descent shape the update live, while the shared base grid never moves. That separation is the whole point: because each task's knowledge lives in its own small adapter rather than in the shared weights, learning Task B can never overwrite what Task A's adapter learned, which is the core trick large language models use to add new skills without catastrophic forgetting or the cost of a full fine-tune.