A toy dialogue manager over a small synthetic support knowledge base (24 invented snippets). Ask a question, watch retrieval, attention and the reply stage. No real language model runs here.
The science
Each snippet and the question become TF-IDF vectors (word counts weighted by rarity, idf = ln((N+1)/(df+1)) + 1, then length-normalised). Retrieval ranks snippets by cosine similarity:
sim(q, d) = (q . d) / (|q| |d|)
The prompt is system text (60 tokens) + remembered turns (about 18 tokens each) + the top-K snippets + the question, with tokens estimated as 1.3 x words. When it exceeds the window the oldest turns, then the lowest-ranked snippets, are dropped. A transformer layer mixes the question tokens with scaled dot-product attention:
A = softmax(Q K^T / sqrt(d_k)), d_k = 8
The next word is drawn from a softmax with temperature T over candidate words taken from the retrieved snippets (logit = 9 x snippet cosine + 1.5 x idf + 2.5 if the word is in the question; a made-up scoring rule for illustration):
p_i = exp(z_i / T) / sum_j exp(z_j / T)
H = - sum p_i log2 p_i
Low T concentrates probability on the best candidate (low entropy); high T flattens it. The panel at the bottom right of the 3D view draws the attention heat map (rows sum to 1) and these probabilities for the last question.
Honest simplifications: the embeddings in the attention map are hash-seeded toy vectors, not trained weights; the reply is an extractive stub (the best snippet quoted back), not generated text; latency is the model 40 + 0.35 x context + 22 x output ms (+180 ms voice), not a measurement; the knowledge base is tiny and synthetic. Real systems use dense neural embeddings, vector indexes and trained transformers.
Checks (computed when the page loads):