LoRA 2D: Low-Rank Adapters for Fine-Tuning

Low-Rank Adaptation (LoRA) is the technique that makes it practical to fine-tune huge language models on modest hardware: instead of updating every weight in a d×k matrix, it freezes the original weights and learns a small correction factored as ΔW = (α/r)·B·A, where B and A are skinny d×r and r×k matrices. This 2D simulator runs the exact same live gradient descent as its 3D companion, training B and A to approximate a fixed rank-8 target correction, but renders both matrices as flat heatmaps alongside a scrolling loss-curve panel so you can watch convergence as a graph instead of a 3D bar chart. Sweep the rank slider below and above the target's true rank to see the reconstruction-loss plateau appear and disappear on the curve, and watch the trainable-parameter and memory-savings readouts show exactly why a rank-4 or rank-8 adapter needs a small fraction of a full fine-tune's parameters.