🧠 3D Attention Cost: Sliding-Window, Strided Sparse and Linear-Kernel Attention Against Full Softmax in Error, Flops and Memory
An attention matrix built as a bar field on a Blender accelerator card: full causal softmax against sliding-window with sink tokens, strided sparse and elu+1 linear attention, with counted multiply-adds, output error, kept softmax mass, flops at long context and the naive score memory in a glass tank, checked by closed forms, causality and a recurrent-versus-matrix identity.
An attention matrix built as a bar field on a Blender accelerator card: full causal softmax against sliding-window with sink tokens, strided sparse and elu+1 linear attention, with counted multiply-adds, output error, kept softmax mass, flops at long context and the naive score memory in a glass tank, checked by closed forms, causality and a recurrent-versus-matrix identity.
3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install