Home▸Robotics & Kinematics▸Vision-Tactile Fusion for Grasping in 2D: Side-View Sensor Blend

Vision-Tactile Fusion for Grasping in 2D: Side-View Sensor Blend

Interactive 2D side-view robot gripper: a noisy, occasionally-occluded overhead camera and a precise-but-contact-only fingertip sensor each estimate where an object sits, and inverse-variance fusion blends them into a live grasp target. Compare fused, vision-only and tactile-only strategies over repeated descend-contact-grasp trials with a scrolling error trace and trial-error bar history.

Robotics & Kinematics2DModerate60 FPS⇄ 3D version
2d-robotics-topic-11 ↗ Open standalone

A robot gripper must find an object it cannot yet touch. An overhead camera gives a continuous but noisy, occasionally-occluded, possibly-miscalibrated horizontal position estimate; a tactile sensor on the fingertip gives a precise estimate the instant it makes contact, but is blind until then. This side-view 2D rendition draws the full sense-approach-contact-grasp cycle as a vertical descent against a horizontal position axis, fuses both estimates by inverse-variance weighting, and adds a scrolling error trace plus a per-trial error-bar history so you can watch exactly how fast each strategy converges — not just whether the final trial succeeded. Compare a fused strategy against relying on either sensor alone and see how grasp success rate responds to noise, calibration bias and occlusion.

⚙ Under the hood

Interactive 2D side-view robot gripper: a noisy, occasionally-occluded overhead camera and a precise-but-contact-only fingertip sensor each estimate where an object sits, and inverse-variance fusion blends them into a live grasp target. Compare fused, vision-only and tactile-only strategies over repeated descend-contact-grasp trials with a scrolling error trace and trial-error bar history.

roboticssensor fusiontactile sensingcomputer visiongraspingcalibration

2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)