AI Escalation Policy Learner — Contextual Bandit for Human Handoff

Policy Parameters

Exploration rate ε 15%

Learning rate α 0.20

Ticket arrival rate 2.0/s

Human escalation cost 0.30

Live Readouts

Tickets processed

Escalation rate

Avg reward (last 40)

—

Cumulative reward

0.0

Legend

AI auto-resolve Q-value bar

Human escalation Q-value bar

Ticket resolved with high reward

Ticket resolved with low reward

How it works

Every incoming ticket has a hidden type (Billing, Technical, Refund, Complaint, General). The router is a contextual bandit: for state (ticket type) s it keeps a running value estimate Q(s, a) for each action a ∈ {AI, Human}.

ε-greedy policy:
  with prob ε      → pick a random action (explore)
  with prob 1-ε    → pick argmax_a Q(s,a)   (exploit)

Value update after observing reward r:
  Q(s,a) ← Q(s,a) + α · [ r − Q(s,a) ]

Reward model (hidden ground truth the bandit must discover):
  r_AI(s)    = satisfaction_AI(s) + noise
  r_Human(s) = satisfaction_Human(s) − cost_weight + noise

ε (exploration) — higher values keep sampling both actions even after the policy has a favourite, at the price of more low-reward tickets in the short run.
α (learning rate) — how fast each new observation overwrites the running estimate; too high and the bars jitter, too low and the policy converges slowly.
Escalation cost — the router is penalised whenever it hands a ticket to a human agent, modelling staffing cost; push it high enough and the policy will happily accept lower satisfaction to keep tickets automated.
Ticket types such as Complaint and Technical have low true AI satisfaction, so with a moderate cost the bandit should learn to escalate them while auto-resolving General and Billing.

Real-world relevance: this is the same explore/exploit trade-off that sits behind live AI-triage systems — a support platform must keep testing whether the model has gotten good enough to take a category off human agents, without tanking satisfaction while it experiments.

Ticket types: Billing · Technical · Refund · Complaint · General Blue bars = AI value · Amber bars = Human value

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.