Discriminative Fine-Tuning: Layer-Wise Learning Rates

When you fine-tune a pretrained network on a new task, applying one learning rate to every layer is rarely optimal: deep general-purpose features can be wrecked by a rate that's perfect for a freshly initialised task head. This simulator renders a pretrained backbone stacked beneath a new classification head as glowing 3D layers, and drives every layer's live update pulse from the actual discriminative learning-rate formula used by ULMFiT — a per-layer geometric decay combined with a slanted triangular schedule that warms up fast and decays slowly. Tune the base rate, the discriminative ratio, and how many backbone layers stay frozen, and watch task loss fall while a source-retention readout shows how much of the original pretrained knowledge survives the fine-tuning run.