Home▸Algorithms & AI▸AI Code Generation: Transformer Attention & Token Prediction Visualizer

🤖 AI Code Generation: Transformer Attention & Token Prediction Visualizer

Live simulation of GitHub Copilot / Cursor / Code Llama predicting the next token – multi-head self-attention, live candidate probabilities, and temperature control over softmax(QKᵀ/√d)·V.

Algorithms & AI3DModerate60 FPS📱 Mobile-adapted

Details

CategoryAlgorithms & AI

RenderingThree.js · WebGL

DifficultyModerate

Target FPS60

ControlsTemperature slider (0.1-2.0) reshapes the next-token probability distribution; Generation speed slider (1-10 tok/s) sets streaming rate; Attention heads slider (2-8) sets how many parallel attention bundles are drawn; Context window slider (4-32 tokens) sets how far back attention reaches; Pause/Resume and Reset buttons.

Three.jsAItransformerself-attentioncode generationLLMInstancedMesh

Related simulations

Transformer Self-Attention VisualizerInteractive 3D simulation of transformer self-attention: watch a query token score every key…

Neural Network Architectures: From Perceptrons to TransformersSwitch between Perceptron, MLP, CNN and Transformer architectures and watch a forward pass…

LLM Evaluation & BenchmarksSimulate how LLM benchmark scores (MMLU, HumanEval, BIG-Bench, Chatbot Arena) fluctuate with… ⊞ All 420 in Algorithms & AI →

⚙ Under the hood

Watch a simplified transformer generate code token by token: multi-head self-attention lines connect the token being written to its context window while a live bar chart shows candidate token types competing for the next slot. Temperature reshapes that softmax(QKᵀ/√d)·V-driven distribution — the same mechanism behind GitHub Copilot, Cursor AI and Code Llama.

Three.jsAItransformerself-attentioncode generationLLMInstancedMesh

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install