Autonomous Vehicles: Self-Driving Technology Explained

Levels of Autonomy

SAE J3016 levels: Level 0 (no automation), Level 1 (driver assistance — adaptive cruise control), Level 2 (partial automation — lane keeping + ACC, Tesla Autopilot, GM Super Cruise), Level 3 (conditional automation — vehicle drives in specific conditions, driver as fallback, Mercedes Drive Pilot, Honda Legend), Level 4 (high automation — no driver needed in geofenced area, Waymo, Cruise), Level 5 (full automation — anywhere, anytime, not yet achieved). Current state: L2+ widely deployed, L4 robotaxis operating in limited cities (San Francisco, Phoenix, Beijing). Key challenge: L3 handoff problem — keeping a disengaged human ready to take over in seconds.

Sensors and Perception

LIDAR (Light Detection and Ranging): laser pulses create 3D point cloud, range 200-300m, resolution mm-cm level. Types: mechanical spinning (Velodyne VLP-16), solid-state (Luminar Iris, Innoviz), FMCW (Aeva — measures velocity directly). Cameras: 8-12 cameras for 360° coverage, deep learning for object detection (YOLO, PointPillars). Radar: 77 GHz automotive radar, works in rain/fog/night, measures range and velocity (Doppler). Ultrasonic: short range (<5m) for parking assistance. Sensor fusion: combining data from multiple sensors for robust perception — early fusion (raw data), late fusion (decision-level), mid-level fusion. HD Maps: centimeter-accurate pre-mapped environment (lane markings, signs, traffic lights) — used by Waymo, not by Tesla (vision-only approach).

Path Planning and Control

Perception → Prediction → Planning → Control pipeline. Prediction: forecasting trajectories of other agents (vehicles, pedestrians) — probabilistic models, graph neural networks. Motion planning: route planning (global path), behavior planning (lane change decision), trajectory planning (smooth, collision-free path). Algorithms: A* for graph search, RRT (Rapidly-exploring Random Trees) for sampling-based planning, lattice planners, optimization-based (Model Predictive Control). Control: lateral control (steering — Stanley method, Pure Pursuit), longitudinal control (speed — PID, MPC). End-to-end learning: neural network directly maps sensor input to control output (bypassing modular pipeline) — Tesla's approach, Wayve. Safety: redundant systems, watchdog processors, minimum risk condition (pull over safely).

V2X and Infrastructure

V2X (Vehicle-to-Everything): V2V (vehicle-to-vehicle), V2I (vehicle-to-infrastructure), V2P (vehicle-to-pedestrian). DSRC (Dedicated Short-Range Communications): 5.9 GHz, 802.11p, low latency (~2ms). C-V2X (Cellular V2X): 3GPP standard, uses LTE/5G sidelink. Applications: intersection collision warning, emergency vehicle priority, cooperative perception (sharing sensor data between vehicles). Smart infrastructure: intelligent traffic lights, dynamic speed limits, connected work zones. Digital twins: city-scale simulation for AV testing (NVIDIA DRIVE Sim, CARLA, Waymo's SimulationCity). Regulation: FMVSS adaptation for vehicles without steering wheels, liability frameworks, data privacy.

Challenges and Future

Edge cases: unusual scenarios not well-represented in training data (construction zones, unusual objects, adversarial conditions). Long tail problem: handling rare but critical situations — 10⁹ miles of testing for statistical significance at human-level safety. Weather: rain, snow, fog degrade LIDAR and cameras — thermal cameras, 4D imaging radar as solutions. Ethical dilemmas: trolley problem in practice — current approach: follow traffic laws, minimize overall risk. Economic impact: $4 trillion mobility market, disruption of trucking (1.7M US jobs), ride-hailing, insurance. Environmental: fleet optimization, platooning (reduced air resistance), potential increase in VMT (induced demand). Timeline: L4 robotaxis expanding to more cities (2025-2030), L4 trucking on highways (2026-2028), L5 remains decades away.

Frequently Asked Questions

What are the levels of autonomous driving?

SAE defines 6 levels (0-5): from no automation to full automation. Currently, Level 2 (partial automation) is widely available, Level 4 (high automation in limited areas) is operating in select cities, and Level 5 has not been achieved.

What sensors do self-driving cars use?

Self-driving cars typically use a combination of LIDAR (3D laser scanning), cameras (visual recognition), radar (all-weather detection), and ultrasonic sensors (close-range), fused together for robust perception.

What is sensor fusion?

Sensor fusion combines data from multiple sensor types (LIDAR, cameras, radar) to create a more accurate and reliable understanding of the environment than any single sensor could provide.

Are self-driving cars safe?

Autonomous vehicles have shown improved safety in controlled environments — Waymo reports significantly fewer crashes than human drivers. However, edge cases and rare scenarios remain challenging.

What is V2X communication?

V2X (Vehicle-to-Everything) enables vehicles to communicate with other vehicles, infrastructure, and pedestrians using wireless technology, improving safety and traffic efficiency.

Try it live

Everything above runs in your browser — open Self-Driving Car Sensor Fusion & Path Planning Simulator and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.