🧭 2D Eligibility Traces: TD(lambda) Credit Assignment, Forward and Backward View on a Sparse-Reward Grid World
Four value heat maps on a sparse-reward grid world (exact Bellman solution and TD(0), TD(lambda), TD(1) learners with glowing eligibility traces), RMSE and V(start) plots, and seven independent checks (Bellman solve, forward = backward view, TD(0) and Monte Carlo limits, trace decay, rollouts, learning speed).
⚙ Under the hood
Four value heat maps on a sparse-reward grid world (exact Bellman solution and TD(0), TD(lambda), TD(1) learners with glowing eligibility traces), RMSE and V(start) plots, and seven independent checks (Bellman solve, forward = backward view, TD(0) and Monte Carlo limits, trace decay, rollouts, learning speed).
Eligibility TracesTD LambdaTemporal Difference LearningCredit AssignmentReinforcement LearningLambda ReturnValue FunctionGrid World
2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install