Grammar-Constrained Decoding: Masking an LLM's Token Distribution — 2D

Structured-output modes on modern LLM APIs (JSON mode, function calling, regex- or grammar-constrained generation) work by running a formal grammar alongside the model's own token distribution: at every step, only tokens that keep the output a valid string in the target grammar survive, everything else is masked to zero probability, and the surviving mass is renormalized before sampling. This 2D counterpart renders that per-step distribution as a horizontal probability bar chart over a 20-token toy vocabulary generating a JSON object, with temperature and nucleus (top-p) controls shaping the underlying softmax and a grammar on/off toggle — but it goes further than a single live trajectory: a Monte Carlo batch button replays thousands of complete generations at once and counts, with real numbers, how often unconstrained sampling actually produces valid JSON versus a masked decoder that mathematically cannot fail, plus a per-step legality histogram showing how that failure rate responds to temperature.