Eligibility Traces: TD(λ) Credit Assignment

Reinforcement-learning agents face a hard question: when a reward finally arrives, which of the many earlier states and actions actually deserve credit for it? TD(λ) answers that with an eligibility trace — a decaying memory of recently visited states — that spreads each step's temporal-difference error backward across the trajectory. This simulator runs a live grid-world agent under TD(λ) prediction in real 3D: each cell's height and colour track its learned state-value V(s), a glowing ring marks cells with an active eligibility trace, and the λ slider lets you slide continuously between one-step TD(0), which only ever credits the very last transition, and something close to full Monte-Carlo credit assignment, where nearly the whole episode shares in the outcome.