Bidirectional RNN: Forward-Backward Context Fusion

Sequence

Sequence length 7 tokens

Playback speed 1.0 steps/s

Inspect position

Token index t = 3

Global step

0 / 7

Context status

pending

‖h forward‖

0.000

‖h backward‖

0.000

‖context‖ = ‖[h_fwd ; h_bwd]‖

0.000

Forward pass h→ (reads left → right) Backward pass h← (reads right → left) Fused context [h→ ; h←] Input token embedding x_t

How it works

A bidirectional RNN runs two independent recurrent chains over the same input sequence and concatenates their hidden states at every position:

h_t→ = tanh(W_hh→ h_(t-1)→ + W_xh→ x_t + b→)   (forward: t = 0 … T-1)
h_t← = tanh(W_hh← h_(t+1)← + W_xh← x_t + b←)   (backward: t = T-1 … 0)
y_t  = [ h_t→ ; h_t← ]                          (context available at t)

The forward chain only ever sees tokens 0…t (its hidden state carries the past). The backward chain only ever sees tokens t…T-1 (it carries the future). Neither chain alone knows both — only the concatenation y_t does, which is why bidirectional RNNs beat plain forward RNNs on tasks like named-entity recognition where a word's meaning depends on what comes after it too.

Play / Step — both heads advance one token per step, forward scanning inward from the left, backward scanning inward from the right, exactly as the two chains process independently in real time.
Context fusion bridge — a green link between the forward and backward spheres at position t lights up only once both heads have reached t — so the middle of the sequence fuses first, the two ends fuse last (each end has to wait for the opposite chain to travel the full sequence length).
Mode toggle — "Forward-only" disables the backward chain entirely, showing the same sequence with a standard unidirectional RNN: no green fusion ever appears because there is no future context to merge.
Inspect slider — reads the live hidden-state norms ‖h→‖, ‖h←‖ and the fused ‖[h→;h←]‖ at any position, before or after fusion completes.

Real-world relevance: this is the exact mechanism inside BiLSTM/BiGRU encoders used for part-of-speech tagging, named-entity recognition and the encoder half of many sequence-to-sequence translation models.

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.