Self-Attention Simulator

Transformer language models like BERT and GPT understand a sentence by letting every token look at every other token before deciding what it means. This simulator renders that self-attention step directly: pick a sentence, and watch each token compute a query/key dot-product score against the others, normalize the row into a probability distribution with softmax, and blend value vectors accordingly. The heatmap in the middle is the live attention matrix — brighter cells mean stronger attention — while the token graph below animates the flow of information as edges whose thickness tracks the current weight. Adjust temperature and key noise to see how sharply or loosely the model can disambiguate an ambiguous word like "bank" from its neighbors.