Multimodal LLM Cross-Attention Simulator
Interactive cross-modal attention simulator: watch text tokens query an image-patch grid through scaled dot-product attention, fusing visual and language tokens into a shared embedding space, with temperature, embedding noise and patch resolution tunable in real time.
Multimodal large language models understand images and text together by projecting both into a shared embedding space and letting the two token streams attend to each other. This simulator visualizes that fusion step directly: an image encoder's patch grid produces visual tokens, a text encoder produces language tokens, and cross-attention computes, for every text token, a similarity-weighted distribution over every visual token. Adjust the patch resolution, softmax temperature, embedding noise and token count to see how sharply — and how correctly — language grounds itself in the right region of the image.
On this 2D canvas, a text token queries every image patch. The strength of their interaction is shown through weighted links and live softmax/entropy readouts.
2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install