Grouped-Query Attention: Shrinking the KV Cache

Every large language model has to keep a running cache of key and value vectors for every token it has already generated, and in plain multi-head attention that cache grows with the number of attention heads. Grouped-Query Attention (GQA) — used in LLaMA-2/3, Mistral, Gemma and most modern serving stacks — lets many query heads share one key/value head instead of owning their own, cutting the cache size (and the memory bandwidth needed to read it every token) by a large, tunable factor. This 2D diagram lays out the head-sharing topology directly: a row of query-head blocks on top, a row of shared KV-cache blocks below sized to their real byte footprint, animated flow dots connecting each query head to its group, and a live chart sweeping the group count from MQA to MHA so you can see the exact linear memory relationship while you tune head count, group count and cached sequence length.