MDP parameters

Discount factor γ: 0.90 Transition reliability P(intended): 0.80

Remaining probability is split evenly between veering 90° left and 90° right of the intended action — a genuine stochastic transition model, not deterministic movement.

Reward function

Step cost: -0.04 Obstacle bump penalty: -0.75

Obstacle layout

Click any cell in the grid to toggle it as an obstacle (start/goal locked).

Value iteration

Convergence

Sweep #0

Max Δ this sweep—

Statusnot started

V(start)—

Optimal path length—

Every sweep applies the Bellman optimality backup V(s) ← max_a Σ P(s'|s,a)[R(s,a,s') + γV(s')] synchronously to all non-terminal states. The chart below plots the max value-change per sweep, which shrinks toward zero as V converges to V*.