A human "expert" demonstration traces one smooth reference path from the start pose to the goal (the pale dashed curve). A policy trained purely by copying that single trajectory only ever saw states on the path — so the moment per-step noise nudges it off the curve, it is now in a state the training data never covered, and its next action is a guess. That guess tends to be worse than an on-distribution action, pushing it further off — and each subsequent action is evaluated from an even more unfamiliar state. That feedback loop is what compounds a tiny per-step error into a large final miss.
naive: offset += noise · (1 + gain·|offset|) · dt
dagger: offset += noise · dt
if |offset| > threshold: offset *= (1 − correction)
- Per-step noise — how imprecise each individual action is; present in both policies equally.
- Compounding gain — how much worse the naive policy's actions get as it drifts further from states it was trained on (the distribution-shift penalty). Set to 0 to see noise alone, with no compounding.
- DAgger correction strength — how hard the corrective label pulls the DAgger policy back toward the reference once it drifts past the "off-distribution" threshold (the dashed ring around the executed point). This models the demonstrator periodically re-labeling the correct action from states the rollout actually visits.
- Auto-run — repeats the trajectory many times unattended so the drift chart accumulates enough runs to show the two policies' typical behavior, not just one lucky or unlucky rollout.
Real-world relevance: this is why production robot-manipulation policies are rarely trained from a single batch of demonstrations. DAgger-style pipelines deploy an early policy, let it drift into unfamiliar states, and have the human label the correct action from exactly those states — closing the train/test distribution gap that pure behavior cloning cannot fix on its own.