Home▸AI & Machine Learning▸3D Attention Cost: Sliding-Window, Strided Sparse and Linear-Kernel Attention Against Full Softmax in Error, Flops and Memory

🧠 3D Attention Cost: Sliding-Window, Strided Sparse and Linear-Kernel Attention Against Full Softmax in Error, Flops and Memory

An attention matrix built as a bar field on a Blender accelerator card: full causal softmax against sliding-window with sink tokens, strided sparse and elu+1 linear attention, with counted multiply-adds, output error, kept softmax mass, flops at long context and the naive score memory in a glass tank, checked by closed forms, causality and a recurrent-versus-matrix identity.

AI & Machine Learning3DAdvanced60 FPS📱 Mobile-adapted🟠 Blender⇄ 2D version
3D Attention Cost: Sliding-Window, Strided Sparse and Linear-Kernel Attention Against Full Softmax in Error, Flops and Memory ↗ Open standalone
⚙ Under the hood

An attention matrix built as a bar field on a Blender accelerator card: full causal softmax against sliding-window with sink tokens, strided sparse and elu+1 linear attention, with counted multiply-adds, output error, kept softmax mass, flops at long context and the naive score memory in a glass tank, checked by closed forms, causality and a recurrent-versus-matrix identity.

Sparse AttentionLinear AttentionSliding WindowTransformerAttention SinkComputational CostLong ContextMachine Learning

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)