Grammar-Constrained Decoding: Masking an LLM's Token Distribution
Interactive 3D visualizer for grammar-constrained (structured-output) decoding: watch a language model's per-step token distribution get masked down to only grammar-valid tokens, renormalized, and sampled, with temperature, nucleus (top-p) and a grammar on/off toggle to compare against unconstrained sampling.
Structured-output modes on modern LLM APIs (JSON mode, function calling, regex- or grammar-constrained generation) work by running a formal grammar alongside the model's own token distribution: at every step, only tokens that keep the output a valid string in the target grammar survive, everything else is masked to zero probability, and the surviving mass is renormalized before sampling. This simulator renders that per-step distribution as a ring of 3D bars over a small 20-token toy vocabulary generating a JSON object, with temperature and nucleus (top-p) controls shaping the underlying softmax, and a grammar on/off toggle that lets you watch unconstrained sampling drift into invalid JSON — the exact failure mode grammar-constrained decoding is built to eliminate.
Watch a language model's per-step token probability distribution get masked down to only grammar-valid tokens, renormalized, and sampled — the mechanism behind JSON-schema and function-calling structured-output modes — with temperature, nucleus (top-p) and a grammar on/off toggle to compare against unconstrained sampling.
3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install