Operant & Classical Conditioning Lab

Conditioning Trial

REINFORCED

Learning rate α 0.30

Reward magnitude λ 1.00

Trial #

Assoc. strength V

0.00

ΔV last trial

–

Prediction error

–

Associative strength (V)

Each trial: CS (light) appears, the agent runs the maze, and V is updated by the Rescorla-Wagner rule. In extinction mode the reward is withheld, so V decays toward zero.

How it works

The T-maze pairs a conditioned stimulus (the light at the choice point) with an unconditioned outcome (food reward at the end of the arm). On every trial the agent runs from the start box to the choice point, the light cue turns on, and — while reinforcement is active — a reward is delivered at the arm. The associative strength V between light and reward is updated after each trial using the Rescorla-Wagner rule.

V_new = V_old + α · (λ − V_old)
α = learning rate     λ = asymptotic reward value
(λ − V_old) = prediction error

Learning rate α — how large a step V takes per trial; high α produces fast but noisier learning, low α produces slow, smooth acquisition.
Reward magnitude λ — the ceiling V asymptotically approaches under reinforcement; a bigger reward supports a stronger learned association.
Extinction — withholding the reward sets the effective λ to 0, so the same rule now drives V back down: the light no longer predicts food, and the learned association fades.
Prediction error (λ − V_old) — the core teaching signal: learning is fastest when the outcome is most surprising, and shrinks to zero once the agent's expectation matches reality.

Real-world relevance: this same delta-rule shows up across behavioral neuroscience — from Pavlov's salivating dogs to dopamine reward-prediction-error signals in the midbrain that update value estimates in operant learning.

Agent Light cue (CS) Reward (US)

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.