Ego speed--
True gap--
Tracked gap--
Brakeidle
Loading scene...
lidar return
radar return
camera estimate
rejected by gate
fused track (box + 2σ bar)
safe-distance line
Loading Blender-built road scene...

Autonomous Vehicle Sensor Fusion Lab Blender 5.2

A self-driving car tracks the car ahead with lidar, radar and camera, fuses them in a Kalman filter and decides when to brake. Change noise, weather, latency or occlusion and watch the safety margin.
Scenario presets

Sensors and fusion

Per-sensor 1σ range noise at the current gap and weather. Camera depth noise grows with range squared; lidar and camera degrade in rain and fog, radar barely does.

Scenario parameters (restart on change)
Live readouts

Range to the lead car: truth, fused estimate with a 2σ band, and every measurement (filled = accepted, red ring = rejected by the gate).

Ego speed, true lead speed and the speed the tracker believes. The lag after the lead starts braking is the cost of a constant-velocity model plus latency.

Braking budget at the start speed: distance travelled during perception latency, actuation delay and braking, against the gap available if the lead stopped dead.

Experiments (Monte Carlo, 60 seeded runs each)
Runs the current scenario headlessly with different seeds. Tables show RMS tracking error, collision rate, phantom braking (braking before the lead does anything) and minimum gap.
Independent numerical checks
The science

Sensor models

Each sensor returns the range z = r + n with Gaussian noise n ~ N(0, σ²). Lidar: σ = 0.06 m (times the weather factor). Radar: σ = 0.35 m and a direct range-rate (Doppler) ρ̇ = u - v_ego with σ = 0.10 m/s. Camera depth: σ = 0.15 + 0.0009 r² m, because depth from image size or stereo disparity degrades quadratically with range. Each sensor also misses detections with some probability, worse for lidar and camera in rain and fog. The numbers are typical orders of magnitude, not a datasheet of any product.

Kalman filter

State x = [p, u] (lead position and speed in the road frame), constant-velocity model x⁺ = F x, F = [[1, Δt],[0, 1]], process noise for unknown lead acceleration Q = σa² [[Δt⁴/4, Δt³/2],[Δt³/2, Δt²]]. Each measurement is a scalar update: y = z - Hx, S = H P Hᵗ + R, K = P Hᵗ / S, x ← x + K y, P ← (I-KH) P (I-KH)ᵗ + K R Kᵗ (Joseph form). Radar range uses H = [1, 0], Doppler H = [0, 1]. Sensors with different rates are fused by applying their updates one after another.

Steady-state gain (the analytic check)

For a position-only sensor with period T the covariance recursion converges to a fixed point. With tracking index Λ = σa T² / σz the gains are α = -(Λ² + 8Λ - (Λ+4)√(Λ²+8Λ))/8 and β = (Λ² + 4Λ - Λ√(Λ²+8Λ))/4 (the alpha-beta filter of Kalata, 1984). The page iterates the real filter code until it settles and compares both gains with these closed forms.

Gating and false returns

A measurement is accepted only if its normalised innovation d² = y²/S is below 9 (3σ; about 99.7 % of true returns pass). Occluded lidar/camera returns hit the truck instead of the lead car, and radar ghosts come from guard rails: both are far outside the gate and are rejected. Without the gate they drag the track toward the wrong range, and the ego car brakes for a car that is not there (phantom braking). The gate has a price: an unexpectedly hard manoeuvre can also fall outside it, so the track is re-initialised after 0.8 s without an accepted update.

Latency

Every measurement arrives L seconds after it was taken. With timestamp compensation the update uses the model p(t_meas) = p(now) - u L (H = [1, -L]) and the odometry at the measurement time. Without it the stale range is simply added to the current ego position. Compensation by constant-velocity extrapolation helps in steady closing but cannot know about an acceleration that happened inside the delay, so the benefit shrinks during hard braking.

Braking distance and the decision rule

Stopping distance d = v (tlat + tact) + v² / (2 μ g): the car keeps its speed during delays, then decelerates at μg (capped at 9.5 m/s²). The ego car brakes when the tracked gap falls below the worst-case safe distance in the spirit of the published RSS idea: dsafe = v τ + v²/(2 ae) - u²/(2 alead) + 2 m, where the lead is assumed able to brake at 8 m/s². Once triggered the brake is latched and reaches full force after the actuation delay plus a 0.15 s ramp.

Occlusion

A large vehicle in the next lane is modelled as blocking the line of sight of lidar and camera for as long as it lies between the ego car and the lead car (a stand-in for a bend or lane offset). During that time those sensors return the truck's range or nothing; radar partly sees through multipath, with a lower detection rate and 2.5 times the noise.

Simplifications

One dimension along the lane, one tracked object, known ego odometry, no data association, no clutter other than the modelled ghosts, instantaneous steering is not modelled, tyre grip is a single friction coefficient, and no sensor calibration error. The Monte Carlo tables are statements about this model, not about any real vehicle. Real systems use many more tricks (interacting multiple models, association, learned detectors); this page shows the underlying trade-offs. Algorithm names for further reading: Kalman filter (Bar-Shalom et al., Estimation with Applications to Tracking and Navigation), alpha-beta tracking index (Kalata 1984), RSS (Shalev-Shwartz et al. 2017); check the originals for exact statements.

Independent educational model; it does not represent the perception system of any specific vehicle or company.

Made with 🟠 Blender · Three.js