Robot Language Disambiguation 2D: Ask vs. Act

Vision-Language-Action robots don't just execute commands — they must first ground a referring phrase like "pick up the red block" in the objects they actually see, and decide whether they're confident enough to act. This top-down 2D simulator renders a table of candidate objects and a robot arm, computes a live grounding-confidence score for every object against the instruction's revealed descriptors, and applies a threshold-gated ask-vs-act policy: when the confidence gap between the top two candidates is wide enough the robot grasps immediately, and when it's too close to call the robot asks a clarifying question that reveals one more descriptor before trying again. Drag the table view to pan and scroll to zoom, and watch the descriptor match matrix light up as each attribute is revealed. Tune the threshold, the number of descriptors given upfront, and the candidate count to see how referential ambiguity trades off against dialogue latency — the exact confidence-gating problem behind real VLA policies like RT-2 and OpenVLA.