AR overlay registration error (2D plan view)

Top-down plan of a headset and a square marker: the pose-noise cloud is long in depth and thin sideways, and head-motion latency drags the overlay off the marker.

Switch to 3D

Presets

Parameters

Model checks

The science

What is modelled

A headset camera with focal length f (in pixels) tracks a flat square marker of side S at distance Z. The tracker finds the four marker corners with Gaussian pixel noise σ and turns them into a pose. The virtual overlay is then drawn from that pose, and the picture reaches the eye τ seconds after the head pose was measured.

Equations

  • Pinhole projection: a point at lateral offset X and depth Z lands at u = f X / Z pixels. One pixel therefore stands for Z/f metres at the marker.
  • Apparent side length s = f S / Z pixels, so depth is recovered as Ẑ = f S / s̄ where s̄ is the mean of the four measured sides.
  • Error propagation: each side length is a difference of two noisy corners (std σ√2); the four sides share corners, and the mean has variance σ²/2. Hence σZ = Z² σ / (√2 f S). Depth error grows with the square of distance.
  • Centre (lateral) error from the mean of four corners: σX = Z σ / (2 f).
  • Latency: with head yaw θ(t) = A sin(2π fh t) the overlay is drawn with the stale angle θ(t - τ). The angular error is A[sin ωt - sin ω(t-τ)], whose amplitude is exactly 2A |sin(ωτ/2)| ≈ ω A τ. At the marker it is displaced by about Z times that angle.

Checks run in this page

  • Monte Carlo depth/centre noise: 20,000 random corner sets are turned into Ẑ and compared with the formulas above (within 4 %, valid while σ is small compared with the side s in pixels).
  • Latency amplitude: the delayed sine difference is scanned numerically over five periods and compared with 2A|sin(ωτ/2)|.
  • Pixel scale: a point is projected through a real Three.js perspective camera with the same focal length, and the result must equal f X / Z pixels (independent of the formula used above).

Honest simplifications

  • The marker is seen front-on and only the size-based depth estimate is analysed. Real PnP solvers also have a strong tilt/depth coupling for oblique markers, which is not modelled.
  • Corner noise is white Gaussian and independent; real detectors have correlated, lighting-dependent errors, and cameras have distortion that is ignored here.
  • Latency is a pure delay of the head angle with a prescribed sinusoidal motion. Real devices partly cancel it with prediction and late-stage reprojection, which is not simulated.
  • The orange offsets in the view are multiplied by the "visual exaggeration" slider so that millimetres are visible; the numbers in the read-outs are the true values.

What the 2D view adds

  • The plan view shows that pose noise is an ellipse: depth error σZ is much longer than lateral error σX, which a perspective 3D camera hides. The dots are fresh Monte Carlo pose estimates drawn at 30 Hz.
  • The camera-image panel is a zoomed crop of the marker: the four corner detections scatter by σ pixels (drawn × the exaggeration slider).

The headset, marker board and table sprites and the baked wood background are Blender 5.2 renders of the same models used in the 3D version; the tracking noise, plan view and overlay are computed live in your browser on one shared maths module (mr_core.js).

Loading sprites...
Overlay lag at the marker (mm) over the last 4 s
Pose noise against distance Z (log-log)
Overlay lag peak against latency
Made with 🟠 Blender