Every sphere is one token placed at a fixed point in a 3D embedding space (a Fibonacci lattice on a sphere, so nearby points stay spread out). Once per second a new query token "asks a question" of every other token; the brightness and thickness of each edge show how much attention that query pays to that key.
Attention(Q,K,V) = softmax( QK𝖣 / √d_k ) V
score(i,j) = content(i,j) − λ · dist(i,j)
weight(i,j) = softmax_j( score(i,j) / τ )
- Tokens N — sequence length; more tokens means more keys competing for the query's attention mass.
- Temperature τ — softmax sharpness. Low τ makes attention nearly one-hot (peaky, almost hard-argmax); high τ spreads weight almost uniformly across all keys.
- Locality bias λ — a positional penalty proportional to sequence distance, the way real models use rotary/relative position encodings to prefer nearby context; raising it pulls attention toward neighbouring tokens.
This mirrors the core operation inside every transformer layer (GPT, BERT, Claude, LLaMA): queries, keys and values are compared, scaled, passed through softmax, and used to blend value vectors — repeated across many heads and layers to build up context.