Q-learning agent in a 3D gridworld

Episode 0
Step 0
Last return -
Success (50) -
Epsilon -
Greedy path -
Mean V(s) 0
Updates 0
Graph: episode return (dots) and 20-episode average (line).
Hover a tile to inspect its Q-values.
Tabular Q-learning (Watkins & Dayan 1992): Q(s,a) += alpha [r + gamma max Q(s',a') - Q(s,a)], epsilon-greedy exploration. Change the world and watch the agent re-adapt.