3D Multimodal Embedding Space Simulator

Shared Embedding Space

Alignment strength 0.35

Embedding noise 0.20

Concept pairs 6

Avg pair distance

–

Aligned pairs

–

Drift energy

–

How it works

A multimodal LLM does not just attend across token streams — training also pulls image and text encoders toward a single shared vector space, so a picture of a cat and the word "cat" end up as nearby points in the same embedding geometry (the idea behind CLIP-style contrastive alignment). This scene visualizes that space directly: each orange point is an image token's embedding, each blue point is a text token's embedding, and each pair sharing a concept is pulled together by a spring proportional to the alignment strength — exactly what a contrastive loss does over many training steps, compressed into a live physical simulation.

F_pair  = strength · (pos_text − pos_image)
pos    += (F_pair + noise·rand³) · dt
loss   ≈ Σ ‖pos_text − pos_image‖² over pairs

Alignment strength — how hard matched image/text pairs are pulled toward each other, standing in for how well a contrastive objective has trained the shared space.
Embedding noise — random jitter injected into every point every frame, standing in for an under-trained or noisy encoder that hasn't converged.
Concept pairs — how many distinct image/text concept pairs share the embedding space at once.
Pair links — toggle the lines connecting each image token to its true matching text token, so you can watch individual pairs converge or drift apart.

This is mechanically distinct from token-by-token cross-attention: instead of one token dynamically querying another every forward pass, this is the geometry those queries rely on — the learned space where "close" already means "semantically related," across two different modalities.

Image-token embedding Text-token embedding Aligned pair link

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.