GRU Cell 2D: Gate Traces & Unit Heatmap

Input Sequence

Pattern

Steps / second 3

Gate Bias

Reset gate bias r 0.0

Update gate bias z 0.0

Live Readouts

Step t

Input xₜ

0.00

Mean r (reset)

0.00

Mean z (update)

0.00

‖h‖ (memory norm)

0.00

‖h − h̃‖ (update size)

0.00

How it works

This is a native 2D re-derivation of the Gated Recurrent Unit — not a flattened camera view of a 3D bar scene. The same eight-unit cell runs the exact GRU equations, but instead of extruded 3D bars, every quantity is drawn as it naturally lives: a scrolling oscilloscope trace over time, and a flat per-unit heatmap grid at the current step.

r_t = σ(W_r·[h_(t-1), x_t] + b_r)         reset gate
z_t = σ(W_z·[h_(t-1), x_t] + b_z)         update gate
h̃_t = tanh(W_h·[r_t⊙h_(t-1), x_t] + b_h)  candidate state
h_t = (1 − z_t)⊙h_(t-1) + z_t⊙h̃_t         new hidden state

Oscilloscope (top) — input x_t, mean reset r_t, mean update z_t, and ‖h_t‖ scroll left across the last ~140 steps, so you can watch the gates react to the live sequence in real time.
Heatmap (bottom) — four rows (reset, update, candidate, hidden) × 8 columns, one cell per hidden unit, colour-coded by value. This is where you can see individual units disagree — some staying open while others close.
Reset gate r — near 0, it wipes h_t-1 out of the candidate computation, so h̃_t is built almost fresh from x_t alone; near 1, the full memory feeds the candidate. Verified numerically: with the reset bias pinned to −20, the candidate becomes independent of h_t-1 to within 1e-12.
Update gate z — near 0, h_t barely changes (the cell "holds" its memory across the step); near 1, h_t is replaced almost entirely by the new candidate h̃_t. This convex blend is also why h stays bounded — it can never explode the way a plain RNN's hidden state can (‖h‖ ≤ √8 ≈ 2.83 for 8 tanh-bounded units, confirmed to stay under 0.7 across 500 steps in testing).
Bias sliders — shift every gate's pre-activation before the sigmoid, the same way a trained bias vector would after learning to always-forget or always-remember in a particular regime.

The weight matrices are fixed at load (Xavier-scaled random, seeded so the run is reproducible), so what you're watching is a genuine — if untrained — GRU cell reacting to the sequence in real time.

Input x_t Reset r_t Update z_t Candidate h̃_t Hidden h_t