🧩 2D LoRA Low-Rank Adapter: Eckart-Young Rank Limit, Gradient Descent on B and A, and Old-Task Drift
Heat maps of the target weight update, the learned low-rank update and the residual, with the singular spectrum split into kept and lost directions: gradient descent on B and A reaches the Eckart-Young floor, with finite-difference gradient and null-space checks.
⚙ Under the hood
Heat maps of the target weight update, the learned low-rank update and the residual, with the singular spectrum split into kept and lost directions: gradient descent on B and A reaches the Eckart-Young floor, with finite-difference gradient and null-space checks.
LoRALow-Rank AdaptationFine-TuningEckart-YoungSingular ValuesGradient DescentCatastrophic ForgettingParameter-Efficient
2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install