Home▸Machine Learning & Neural Networks▸Vanishing Gradients in RNN Backpropagation

Vanishing Gradients in RNN Backpropagation Through Time

Interactive 3D simulator: watch the gradient signal shrink (or blow up) as it travels backward through an RNN's unrolled time steps, and see how the recurrent weight and activation choice control whether long-range dependencies can be learned at all.

Machine Learning & Neural Networks3DAdvanced60 FPS📱 Mobile-adapted

Sequence models learn by pushing an error signal backward through every unrolled time step of a recurrent network, and each step multiplies that signal by the same local Jacobian — the recurrent weight times the activation's derivative. This simulator runs a real scalar RNN forward, then animates the backward pass step by step in 3D: a row of bars tracks the forward hidden-state magnitude while a second row tracks the live gradient magnitude, shrinking toward the vanishing regime or blowing up toward the exploding one depending on the recurrent weight and activation function you choose. Live readouts report the current step's gradient, the gradient that finally reaches the earliest time step, and how many steps the signal survives above a 10⁻³ threshold — the same exponential mechanism that motivated gated architectures like LSTM and GRU.

⚙ Under the hood

Watch the gradient signal shrink or blow up as it propagates backward through an RNN's unrolled time steps, and see how the recurrent weight and activation choice determine whether long-range dependencies can be learned.

RNNbackpropagationdeep learningsequence modelsgradientsLSTM

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install