Home▸AI & Machine Learning▸3D Lost in the Middle: Softmax Attention with Primacy and Recency Bias, Needle Recall and Context Length

📚 3D Lost in the Middle: Softmax Attention with Primacy and Recency Bias, Needle Recall and Context Length

A Blender-built bookshelf with one book per slice of the context window; the height of each book is the attention it receives and a golden needle book sits at a chosen depth. Primacy and recency bias create a U-shaped recall curve whose floor falls as the context grows, with closed forms checked against sampling.

AI & Machine Learning3DModerate60 FPS📱 Mobile-adapted🟠 Blender⇄ 2D version
3D Lost in the Middle: Softmax Attention with Primacy and Recency Bias, Needle Recall and Context Length ↗ Open standalone
⚙ Under the hood

A Blender-built bookshelf with one book per slice of the context window; the height of each book is the attention it receives and a golden needle book sits at a chosen depth. Primacy and recency bias create a U-shaped recall curve whose floor falls as the context grows, with closed forms checked against sampling.

Lost in the MiddleLong ContextSoftmax AttentionPositional BiasNeedle in a HaystackRetrievalContext WindowLarge Language Models

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)