Gradient Clipping: Exploding & Vanishing Gradients

Deep networks and long RNN unrolls backpropagate a gradient through many layers by the chain rule, and each layer multiplies it by another factor. This simulator renders that chain in 3D as a sequence of nodes whose size and color track the gradient's magnitude on a log scale, animates the actual backward flow of backpropagation from output to input, and lets you tune the network depth and the average per-layer gradient factor to watch the signal explode or vanish exponentially with depth. A global-norm gradient-clipping control — the same mechanism used in PyTorch's clip_grad_norm_ and TensorFlow's clipnorm — shows how rescaling the whole gradient vector once it crosses a threshold keeps deep training numerically stable, with live readouts for the raw norm, the clipped norm, and the scale factor actually applied.