🪟 Sliding-Window Attention with Segment Recurrence: Receptive Field and Cost in 3D
A Blender-built tower of attention layers: choose the window, depth, segment length and segment recurrence, pick a query and a needle token, and see exactly which tokens can reach the query and with what weight, with five independent numerical checks (dense matrix powers, closed forms, variance L(w^2-1)/12).
⚙ Under the hood
A Blender-built tower of attention layers: choose the window, depth, segment length and segment recurrence, pick a query and a needle token, and see exactly which tokens can reach the query and with what weight, with five independent numerical checks (dense matrix powers, closed forms, variance L(w^2-1)/12).
Sliding Window AttentionTransformer-XLSegment RecurrenceReceptive FieldLongformerLong ContextAttention CostTransformers
3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install