Visual-Inertial Odometry: Camera-Corrected AR Tracking

Movement Path

Sensor Settings

IMU Bias Noise 0.40

Camera Update Rate 6 Hz

Scene Feature Density 60%

Playback

Live Readouts

IMU-only error

0.00 m

VIO fused error

0.00 m

Tracked features

0/42

IMU

Fused

How it works

An AR/VR headset tracks its own 6-DoF pose by fusing two very different sensors. The IMU (accelerometer + gyroscope) reports at hundreds of Hz but only measures acceleration and angular rate — position comes from integrating twice, so any small, slowly-wandering bias b in the sensor grows into unbounded position error:

ḃ  = noise           (bias random walk)
v̇_err = b             (bias → velocity error)
ṗ_err = v_err          (velocity error → position error)

Left alone, this dead-reckoning error grows roughly with the cube of elapsed time — the red "IMU-only" ghost in the scene drifts away from the true path and never comes back.

Visual-inertial odometry (VIO) fixes this by tracking fixed feature points in the environment with the camera. Every time enough features are visible (at the Camera Update Rate), the system re-anchors its estimate to those known-fixed points, snapping most of the accumulated error back out — exactly what ARKit, ARCore and HoloLens do dozens of times per second. A textureless environment (low Scene Feature Density, or the headset facing a blank wall) means too few trackable points, the correction is skipped that tick, and the fused estimate briefly drifts like the IMU alone until features come back into view.

IMU Bias Noise — how quickly the raw inertial sensor wanders; higher values make the red ghost diverge faster.
Camera Update Rate — how often a visual correction is attempted; lower rates let more drift build up between corrections.
Scene Feature Density — the fraction of surface points in the room that are texture-rich enough to track; low density means occasional missed corrections.
Violet dots are trackable feature points; the bright ones are inside the device's current field of view and range and are actively correcting the estimate.

Ground truth path IMU-only (drifting) VIO fused (device) Trackable feature

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.