📚 3D Lost in the Middle: Softmax Attention with Primacy and Recency Bias, Needle Recall and Context Length
A Blender-built bookshelf with one book per slice of the context window; the height of each book is the attention it receives and a golden needle book sits at a chosen depth. Primacy and recency bias create a U-shaped recall curve whose floor falls as the context grows, with closed forms checked against sampling.
⚙ Under the hood
A Blender-built bookshelf with one book per slice of the context window; the height of each book is the attention it receives and a golden needle book sits at a chosen depth. Primacy and recency bias create a U-shaped recall curve whose floor falls as the context grows, with closed forms checked against sampling.
Lost in the MiddleLong ContextSoftmax AttentionPositional BiasNeedle in a HaystackRetrievalContext WindowLarge Language Models
3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install