The simulation animates an encoder RNN reading a source sentence word by word into a chain of hidden states, then a decoder generating the translated sentence step by step while an attention heatmap lights up showing which source words the decoder is weighting most heavily at each moment.
Choose a sentence pair from the dropdown, use the speed slider to control how fast the encoder and decoder advance, and watch the attention heatmap update at each decoding step; press Rebuild to load a new sentence pair or Pause to freeze the animation and inspect the current attention weights.
Sentence pair select, playback speed, play/pause, rebuild
The attention mechanism from this exact architecture, originally just a fix for the fixed-size context vector bottleneck in RNN translation, is the direct conceptual ancestor of the self-attention layers powering every modern large language model, after the 2017 Transformer paper removed the recurrence entirely and kept only the attention.
The simulation animates an encoder RNN reading a source sentence word by word into a chain of hidden states, then a decoder generating the translated sentence step by step while an attention heatmap lights up showing which source words the decoder is weighting most heavily at each moment.
The simulation animates an encoder RNN reading a source sentence word by word into a chain of hidden states, then a decoder generating the translated sentence step by step while an attention heatmap lights up showing which source words the decoder is weighting most heavily at each moment.
Choose a sentence pair from the dropdown, use the speed slider to control how fast the encoder and decoder advance, and watch the attention heatmap update at each decoding step; press Rebuild to load a new sentence pair or Pause to freeze the animation and inspect the current attention weights.
The attention mechanism from this exact architecture, originally just a fix for the fixed-size context vector bottleneck in RNN translation, is the direct conceptual ancestor of the self-attention layers powering every modern large language model, after the 2017 Transformer paper removed the recurrence entirely and kept only the attention.