Q-learning hyperparameters

Learning rate α: 0.20 Discount γ: 0.90 Initial exploration ε₀: 1.00 Epsilon decay / episode: 0.965 Minimum ε (floor): 0.03

Arrival rates (veh/min)

North approach: 8 South approach: 8 East approach: 10 West approach: 10

Training

Training speed: 8× decision-steps/frame

View

Live stats

Episode0

Decision step (this ep.)0 / 0

Exploration ε1.00

Avg wait (this ep.)— s

Best avg wait so far— s

Vehicles served (total)0

States visited0 / 1250

Current phaseN–S green

The agent's state is the discretized queue length of each of the 4 approaches (5 buckets each) plus the current signal phase — 1,250 possible states. Its actions are "keep current phase" or "switch phase" (subject to a minimum green time). Every decision step it earns a reward equal to the negative total vehicle-seconds of delay accumulated, and updates a real Q-table with the Bellman equation Q(s,a) += α[r + γ·max Q(s′,a′) − Q(s,a)].