Q-Learning Robot: Reinforcement Learning Simulator

Q-Learning Robot

Learning rate α 0.30

Discount γ 0.90

Exploration decay 0.985

Steps per frame 4

Episode

–

Epsilon ε

–

Avg reward (20 ep)

–

Success rate

–

How it works

The robot occupies one cell of the grid at a time — that cell is its state s. Each step it picks an action (up / right / down / left), mostly the one with the highest learned Q-value, but with probability ε it picks a random action instead to keep exploring. After moving it receives a reward — a small penalty for the step, a large penalty for falling in a pit, a large bonus for reaching the goal — and updates exactly one table entry with the real Q-learning rule:

Q(s,a) += α · ( reward + γ · max(Q(s′,a′)) − Q(s,a) )

α (learning rate) — how much each new experience overwrites the old estimate; too high and learning is noisy, too low and it barely updates.
γ (discount) — how much future reward matters versus the immediate one; higher γ makes the robot plan further ahead.
Exploration decay — ε shrinks by this factor after every episode, so the robot explores randomly at first and increasingly trusts its learned policy later.
Heatmap — each cell's brightness is its best learned Q-value; the arrow shows the current greedy action. Watch the arrows organize into a clean path to the goal as episodes accumulate.

This is the same update rule (tabular Q-learning, Watkins 1989) used to teach real warehouse and mobile robots discrete navigation policies before they graduate to continuous control.

Robot Goal Pit High Q-value