Home▸Algorithms & AI▸AI Code Generation: Transformer Attention & Token Prediction Visualizer

🤖 AI Code Generation: Transformer Attention & Token Prediction Visualizer

Living simulation of how GitHub Copilot / Cursor / Code Llama predict the next token: multi-head self-attention lines, live candidate probabilities and a temperature control over softmax(QKᵀ/√d)·V.

Algorithms & AI3DModerate60 FPS

Details

CategoryAlgorithms & AI

RenderingThree.js · WebGL

DifficultyModerate

Target FPS60

ControlsTemperature slider (0.1-2.0) reshapes the next-token probability distribution; Generation speed slider (1-10 tok/s) sets streaming rate; Attention heads slider (2-8) sets how many parallel attention bundles are drawn; Context window slider (4-32 tokens) sets how far back attention reaches; Pause/Resume and Reset buttons.

Three.jsAItransformerself-attentioncode generationLLMInstancedMesh

Related simulations

3D Boids — Flocking500 agents in 3D space following Reynolds' rules: separation, alignment, cohesion. Emergent…

Ant Colony Pheromone Trails SimulationExplore the fascinating world of ant colonies as you observe how they use pheromone trails…

Pathfinding — A*, Dijkstra, BFSDraw walls, generate mazes and watch A*, Dijkstra, Greedy Best-First and BFS explore the… ⊞ All 343 in Algorithms & AI →

⚙ Under the hood

Watch a simplified transformer generate code token by token: multi-head self-attention lines connect the token being written to its context window while a live bar chart shows candidate token types competing for the next slot. Temperature reshapes that softmax(QKᵀ/√d)·V-driven distribution — the same mechanism behind GitHub Copilot, Cursor AI and Code Llama.

Three.jsAItransformerself-attentioncode generationLLMInstancedMesh

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install