An AR device never "knows" where virtual content should sit — it estimates its own 6-DoF pose each frame from camera + IMU data, then renders the overlay at the anchor's position relative to that estimate. Two error sources make the overlay wobble or lag behind the real world.
Displayed pose: P_disp(t) = lerp( P_disp(t-dt), P_true(t - L), 1 - e^(-dt / τ) )
L = pipeline latency (sensor → pose solve → render), τ = L/3 (smoothing time constant)
P_true is perturbed by tracking noise: P_true = P_anchor + N(0, σ²), σ = sensor noise
Confidence: C = clamp( 100 − 14·σ − 0.05·L − (SLAM ? 8 : 0), 0, 100 ) [%]
- Tracking mode — marker-based locks onto a printed pattern (few points, very stable, works only while the marker is visible); markerless SLAM matches natural features across the whole scene (many points, more coverage, but each match is noisier).
- Sensor noise — per-frame pose-estimation error (camera blur, IMU drift, poor lighting); higher noise makes both the feature cloud and the overlay jitter around the true anchor.
- Pipeline latency — time from capturing a frame to rendering the overlay; higher latency makes the overlay visibly lag behind camera motion ("swimming").
- Feature points — the natural or marker features the tracker is currently matching frame-to-frame; losing too many drops tracking confidence and can lose the lock entirely.
Real AR SDKs (ARKit, ARCore, WebXR) fuse camera vision with gyroscope/accelerometer data through a Kalman-style filter to keep this error under a few millimetres — this simulation exaggerates it so the effect is visible.