Vision-Tactile Fusion for Grasping in 2D: Side-View Sensor Blend
Interactive 2D side-view robot gripper: a noisy, occasionally-occluded overhead camera and a precise-but-contact-only fingertip sensor each estimate where an object sits, and inverse-variance fusion blends them into a live grasp target. Compare fused, vision-only and tactile-only strategies over repeated descend-contact-grasp trials with a scrolling error trace and trial-error bar history.
A robot gripper must find an object it cannot yet touch. An overhead camera gives a continuous but noisy, occasionally-occluded, possibly-miscalibrated horizontal position estimate; a tactile sensor on the fingertip gives a precise estimate the instant it makes contact, but is blind until then. This side-view 2D rendition draws the full sense-approach-contact-grasp cycle as a vertical descent against a horizontal position axis, fuses both estimates by inverse-variance weighting, and adds a scrolling error trace plus a per-trial error-bar history so you can watch exactly how fast each strategy converges — not just whether the final trial succeeded. Compare a fused strategy against relying on either sensor alone and see how grasp success rate responds to noise, calibration bias and occlusion.
Interactive 2D side-view robot gripper: a noisy, occasionally-occluded overhead camera and a precise-but-contact-only fingertip sensor each estimate where an object sits, and inverse-variance fusion blends them into a live grasp target. Compare fused, vision-only and tactile-only strategies over repeated descend-contact-grasp trials with a scrolling error trace and trial-error bar history.
2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install