Eligibility Traces: TD(λ) Credit Assignment (2D)

Reinforcement-learning agents face a hard question: when a reward finally arrives, which of the many earlier states and actions actually deserve credit for it? TD(λ) answers that with an eligibility trace — a decaying memory of recently visited states — that spreads each step's temporal-difference error backward across the trajectory. This 2D simulator runs the same live grid-world agent under TD(λ) prediction: a top-down heatmap tracks each cell's learned state-value V(s), a glowing ring marks cells with an active eligibility trace, a strip chart plots the TD-error δ over time, and the λ slider lets you slide continuously between one-step TD(0), which only ever credits the very last transition, and something close to full Monte-Carlo credit assignment, where nearly the whole episode shares in the outcome.