Home▸AI & Machine Learning▸2D Lost in the Middle: Softmax Attention with Primacy and Recency Bias, Needle Recall and Context Length

📚 2D Lost in the Middle: Softmax Attention with Primacy and Recency Bias, Needle Recall and Context Length

Attention heights over a long context with a needle at a chosen depth: primacy and recency bias, match logit and distractor noise give the U-shaped recall curve, the dependence on context length and sampled retrieval trials, with six independent numerical checks.

AI & Machine Learning2DModerate60 FPS📱 Mobile-adapted🟠 Blender⇄ 3D version
2D Lost in the Middle: Softmax Attention with Primacy and Recency Bias, Needle Recall and Context Length ↗ Open standalone
⚙ Under the hood

Attention heights over a long context with a needle at a chosen depth: primacy and recency bias, match logit and distractor noise give the U-shaped recall curve, the dependence on context length and sampled retrieval trials, with six independent numerical checks.

Lost in the MiddleLong ContextSoftmax AttentionPositional BiasNeedle in a HaystackRetrievalContext WindowLarge Language Models

2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)