Grammar-valid token Masked out (probability = 0) Just sampled
⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.

Grammar-Constrained Decoding: Masking an LLM's Token Distribution

Structured-output modes on modern LLM APIs (JSON mode, function calling, regex- or grammar-constrained generation) work by running a formal grammar alongside the model's own token distribution: at every step, only tokens that keep the output a valid string in the target grammar survive, everything else is masked to zero probability, and the surviving mass is renormalized before sampling. This simulator renders that per-step distribution as a ring of 3D bars over a small 20-token toy vocabulary generating a JSON object, with temperature and nucleus (top-p) controls shaping the underlying softmax, and a grammar on/off toggle that lets you watch unconstrained sampling drift into invalid JSON — the exact failure mode grammar-constrained decoding is built to eliminate.