Perception-Action Loop Latency & the Smith Predictor

A robot chases a moving beacon through a laggy camera: find the delay margin, watch the loop go unstable, then rebuild the present state with a Smith predictor.

Switch to 2D

Presets

Playback

Parameters

Camera

Layers

Quality

Model check

The science

What is modelled

A wheeled robot chases a moving beacon. Its camera and perception software need time, so the controller always works with where the robot was L milliseconds ago (the red ghost in the 3D view). A delay removes phase from the feedback loop without changing its gain; beyond a critical latency the same gains that felt crisp make the robot oscillate and run away. A Smith predictor fights this by replaying the forces the controller has already issued through a model of the robot, which rebuilds an estimate of the present state (the green ghost).

Equations

  • Plant on each axis: m p′′ = F − c p′ + d, controller F = Kp(r − pused) − Kd vused, saturated at 60 N. Without the predictor (p, v)used = (p, v)(t − L) and the characteristic equation is m s² + c s + (Kp + Kd s) e−Ls = 0.
  • Delay margin: let G(s) = (Kp + Kd s) / (s (m s + c)) and ωc the frequency with |G| = 1; the phase margin is PM = π + arg G(jωc) and the loop is stable exactly while L < PM / ωc. Each millisecond of latency costs ωc L radians of phase (second graph).
  • Smith predictor: p̂(t) = Model(p(t−L), v(t−L); F over [t−L, t]), integrated step by step with the model mass and drag. With a perfect model the loop becomes m s² + (c + Kd) s + Kp = 0 for any L; the delay only postpones the reaction to disturbances the model cannot know, such as the push.
  • Model errors (mass, drag sliders) leave P̂ + (P − P̂) e−Ls in the loop, so the margin returns when the model is wrong.

Checks run in the page

  • A time simulation switches from decaying to growing at the latency given by the phase-margin formula.
  • With a perfect predictor and L = 300 ms the simulated regulation equals the closed-form solution of the delay-free second-order system.
  • Replaying the issued forces through an exact model reproduces the true state to rounding error.
  • For a ramp target the steady tracking error equals (c + Kd) v / Kp, reduced by v L without predictor.

Honest simplifications

The robot is a point mass in two independent axes (the drawn orientation only follows the velocity), the delay is a pure constant time shift, the camera noise is uniform and white, and the actuators are ideal apart from the force limit. A real vision pipeline has jitter and dropped frames, and the model of a real robot is never exact: that is why the model error sliders exist. The tracking error of this plain PD law grows with target speed because it has no velocity feed-forward. Numbers are illustrative, not data of a particular machine.

What to try

  • Start at 200 ms without predictor: the status LED turns red, the robot rings and runs away. Lower L below the delay margin and it recovers.
  • Turn the predictor on: the same latency becomes harmless and the second graph's green line shows the margin that remains.
  • Give the predictor a 40 percent wrong mass or a wrong drag and watch the ringing return; then push the robot to see the delayed reaction.
  • Raise the camera noise: large Kd amplifies noise, a trade-off the predictor does not remove.
Loading model...
Loading Blender model...
x position: target, robot, delayed camera view and prediction
Phase margin against latency
RMS tracking error against latency
Made with 🟠 Blender · Three.js