🪜 3D Discriminative Fine-Tuning: Layer-Wise Learning-Rate Ladder, Slanted Triangular Schedule and Forgetting
Two copies of one pre-trained four-layer network are fine-tuned on a shifted task with few noisy labels, one with a single learning rate and one with a layer-wise ladder and slanted triangular schedule: output surfaces rise in 3D while layer slabs glow with their weight drift, checked by finite differences and closed forms.
⚙ Under the hood
Two copies of one pre-trained four-layer network are fine-tuned on a shifted task with few noisy labels, one with a single learning rate and one with a layer-wise ladder and slanted triangular schedule: output surfaces rise in 3D while layer slabs glow with their weight drift, checked by finite differences and closed forms.
Fine-TuningTransfer LearningLearning Rate ScheduleCatastrophic ForgettingBackpropagationULMFiTNeural NetworkMachine Learning
3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install