🧠 Inside an LSTM Cell

About this simulation

A Long Short-Term Memory (LSTM) cell carries a cell state — its long-term memory — down a "highway" from one time step to the next, largely unchanged unless the gates decide otherwise. Three sigmoid gates control that highway: the forget gate decides what fraction of the old memory survives, the input gate decides how much of a new candidate value gets written in, and the output gate decides how much of the (squashed) memory is exposed as the hidden state used for the next prediction — like next month's airline passenger count.

How it works

• Forget gate valve — closes proportionally to (1 − f_t); bars passing through shrink by that amount.
• Input gate valve — scales the incoming candidate value C̃_t before it's added to the forgotten memory.
• New cell state — C_t = f_t · C_{t-1} + i_t · C̃_t, shown as the second bank of bars.
• Output gate valve — h_t = o_t · tanh(C_t), squashing the memory and revealing only part of it as the hidden state.
• The right-hand panel turns the average hidden state into a forecast bump on a toy airline-passenger series.

Controls

• Forget gate — how much of the previous memory is kept (0–100%).
• Input gate — how much of the new candidate is written in (0–100%).
• Candidate value — the raw new content proposed for memory (−1 to 1).
• Output gate — how much of the memory is revealed as hidden state (0–100%).
• Auto time-step — animates the gates through a short sequence automatically.
• ↺ Reset sequence — restarts the forecast trail. Drag to rotate, scroll to zoom.

Did you know?

LSTMs were introduced by Hochreiter & Schmidhuber in 1997 specifically to fix "vanishing gradients" in plain recurrent networks — the additive, gated cell-state highway lets gradients (and memories) flow across hundreds of time steps largely undiminished, which is exactly why they became a workhorse for time-series forecasting.