At every generation step a language model outputs one logit per vocabulary token, turned into a probability with a temperature-scaled softmax:
P(t) = exp(logit(t) / T) / Σ exp(logit(t') / T)
Grammar-constrained decoding (used by JSON-schema / function-calling "structured output" modes in real LLM APIs) runs a formal grammar as a finite-state acceptor alongside the model. At each step only tokens that keep the output a valid string in the grammar are allowed — every other token's probability is forced to zero and the rest are renormalized:
mask(t) = 1 if t ∈ valid-next-tokens(state), else 0
P'(t) = P(t)·mask(t) / Σ P(t')·mask(t')
This 2D counterpart adds a second, independent layer of computation the 3D ring view can't show live: a Monte Carlo batch. Instead of watching one trajectory, it replays thousands of complete 13-step generations at the current temperature/top-p — half with the grammar mask applied, half without — and counts, for real, how many finished as parseable JSON and how the per-step legality rate (chance an unconstrained sample happens to land on a legal token anyway) shifts with temperature.
valid% = 100 × (# outputs where JSON.parse succeeds) / (# runs)
per-step legality[k] = 100 × (# runs where step k's sampled token
was grammar-legal, unconstrained) / (# runs)
- Sample next token — draws one token from the current (possibly masked) distribution and advances the grammar's finite-state machine, which here is producing
{"name":…,"age":…,"active":…}.
- Grammar constraint ON/OFF — with it off the model samples from its raw distribution and can emit a token the grammar forbids, producing invalid JSON — exactly the failure mode structured-output decoding exists to prevent.
- Temperature — low T sharpens the distribution toward the top logit; high T flattens it toward uniform, more entropy per bar, and — as the Monte Carlo batch shows — a slightly higher chance an unconstrained sample accidentally lands on a legal token.
- Top-p — truncates the long tail of implausible tokens before sampling, independent of the grammar mask.
- Monte Carlo batch — run it with grammar OFF at low vs high temperature and compare the "Grammar OFF valid%" stat: it stays near 0% either way at this vocabulary's branching factor, while the per-step legality histogram below the bars still visibly rises with temperature — a real, measured effect, distinct from whether a full 13-token run happens to survive intact.
The 20 horizontal bars are the full toy vocabulary; bar length is that token's current probability, color shows whether the grammar currently allows it, and the highlighted bar is the token that was just sampled.