Home▸Robotics & Kinematics▸Vision-Tactile Fusion for Grasping

Vision-Tactile Fusion for Grasping

A robot gripper estimates an object's position from a noisy, occasionally-occluded camera and a precise-but-contact-only tactile sensor. Watch inverse-variance sensor fusion combine both into a single grasp target and compare it against vision-only or tactile-only strategies over repeated trials.

Robotics & Kinematics3DModerate60 FPS📱 Mobile-adapted
Vision-Tactile Fusion for Grasping ↗ Open standalone

A robot gripper must find an object it cannot yet touch. An overhead camera gives a continuous but noisy, occasionally-occluded, possibly-miscalibrated position estimate; a tactile sensor on the fingertip gives a precise estimate the instant it makes contact, but is blind until then. This simulator runs a full sense-approach-contact-grasp cycle and fuses both estimates by inverse-variance weighting, so you can compare a fused strategy against relying on either sensor alone and watch how grasp success rate responds to noise, calibration bias and occlusion.

⚙ Under the hood

A robot gripper estimates an object's position from a noisy, occasionally-occluded camera and a precise-but-contact-only tactile sensor. Watch inverse-variance sensor fusion combine both into a single grasp target and compare it against vision-only or tactile-only strategies over repeated trials.

Three.jsroboticssensor fusiontactile sensingcomputer visiongraspingcalibration

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)