Home▸AI & Machine Learning▸Q-Learning in a Grid World: How an Agent Learns a Policy

🤖 Q-Learning in a Grid World

A concrete walkthrough of how a reinforcement learning agent uses trial, error and the Bellman equation to discover an optimal path through a simple grid environment.

AI & Machine Learning3DAdvanced60 FPS

A single agent wanders a 3D 6×6 grid, running full Q-learning updates after every real step it takes — no scripted animation, the tile colors and policy arrows you see are literally the live Q-table.

🔬 What It Demonstrates

Every tile's color encodes its current best-known Q-value and every cone shows the greedy action for that state, so you can watch the Bellman equation propagate value backward from the goal through the grid, step by step, episode by episode.

🎮 How to Use

Tune the learning rate, discount factor and exploration rate, then watch the agent bounce between random exploration and exploitation of what it has already learned. Reset the Q-table at any time to start the learning process from zero again.

💡 Did You Know?

Q-learning was introduced by Chris Watkins in 1989 and proven to converge to the optimal policy under mild conditions — even though the agent that's collecting the data is, by design, not always acting optimally.

⚙ Under the hood

A concrete walkthrough of how a reinforcement learning agent uses trial, error and the Bellman equation to discover an optimal path through a simple grid environment.

reinforcement learningq-learninggrid worldagentpolicyartificial intelligencemachine learningThree.js

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install