Sparse & Linear Attention (2D): Taming the O(n²) Transformer Bottleneck

Full self-attention scores every query token against every key token, so its cost scales as O(n²) — double the sequence length and you quadruple the work, which is exactly why long-context transformers can't just run vanilla attention. This 2D simulator renders the query-key attention matrix as a live pan-and-zoom heatmap and lets you switch between four real attention patterns — full, sliding-window (Longformer-style local attention), strided/dilated, and linear (Performer-style random-feature approximation) — while a sequence-length slider and a per-pattern parameter (window radius, stride, or feature rank) reshape the grid in real time. A row-profile panel isolates one token's attention distribution and a log-scale cost panel compares materialized pairs across all four patterns side by side, so the abstract complexity classes in transformer papers become something you can see, drag and inspect under your own hands.