HomeAI & Machine LearningCross-Modal Attention

Multimodal LLM Cross-Attention Simulator

Interactive cross-modal attention simulator: watch text tokens query an image-patch grid through scaled dot-product attention, fusing visual and language tokens into a shared embedding space, with temperature, embedding noise and patch resolution tunable in real time.

AI & Machine Learning2DEasy60 FPS📱 Mobile-adapted⇄ 3D version
multimodal-llm-basics ↗ Open standalone

Multimodal large language models understand images and text together by projecting both into a shared embedding space and letting the two token streams attend to each other. This simulator visualizes that fusion step directly: an image encoder's patch grid produces visual tokens, a text encoder produces language tokens, and cross-attention computes, for every text token, a similarity-weighted distribution over every visual token. Adjust the patch resolution, softmax temperature, embedding noise and token count to see how sharply — and how correctly — language grounds itself in the right region of the image.

⚙ Under the hood

On this 2D canvas, a text token queries every image patch. The strength of their interaction is shown through weighted links and live softmax/entropy readouts.

attentionmultimodalembedding

2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)