Home▸Data Science▸GradNorm Loss Weighting for Multi-Task Learning: Gradient-Norm Equilibrium and Asymmetry

GradNorm Loss Weighting for Multi-Task Learning: Gradient-Norm Equilibrium and Asymmetry

Train four tasks that share one backbone but have very different gradient scales, and let GradNorm adapt the loss weights so the weighted gradient norms match a target set by each task training rate. Change the scale spread, the asymmetry alpha and the learning rates, and compare against the closed-form equilibrium, weight conservation and a finite-difference gradient check. A Blender-built data bench with weights, gradient norms and loss trajectories.

Data Science3DAdvanced60 FPS📱 Mobile-adapted🟠 Blender⇄ 2D version
GradNorm Loss Weighting for Multi-Task Learning: Gradient-Norm Equilibrium and Asymmetry ↗ Open standalone

A shared trunk feeds three task-specific heads, and this simulator lets you switch between the three standard ways a real multi-task network decides how hard each task is allowed to pull on those shared weights: naive equal weighting, Kendall's learned-uncertainty weighting, and Chen et al.'s GradNorm, which actively balances each task's gradient magnitude against its relative training pace. Colored particle streams flow from each task head back into the trunk with a size and speed set by that task's live gradient norm, while side panels track every task's evolving loss weight and gradient norm step by step — so you can watch, in real time, a hard task get starved under equal weighting and then rescued the moment GradNorm or uncertainty weighting takes over.

⚙ Under the hood

Train four tasks that share one backbone but have very different gradient scales, and let GradNorm adapt the loss weights so the weighted gradient norms match a target set by each task training rate. Change the scale spread, the asymmetry alpha and the learning rates, and compare against the closed-form equilibrium, weight conservation and a finite-difference gradient check. A Blender-built data bench with weights, gradient norms and loss trajectories.

multi-task learninggradnormloss weightingdeep learningneural networksmachine learningGradNormMulti-Task LearningLoss WeightingGradient BalancingDeep Learning

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)