Fill-in-the-Middle: AI Code Infilling Simulator

Snippet

FIM Input Ordering

Sampling temperature T 0.40

Generation State

Middle token

0 / 6

Match vs ground truth

—

Perplexity so far (2^H)

—

prefix suffix special token unfilled hole matched pick diverged pick

How it works

Code models like Codex, StarCoder, Code Llama and DeepSeek-Coder are trained not only to continue code left-to-right, but to infill a gap inside existing code — exactly what "accept suggestion" does when your cursor sits between two lines you already wrote. Training reformats each example as a linear sequence with special tokens, most commonly PSM (Prefix-Suffix-Middle):

<PRE> prefix_tokens <SUF> suffix_tokens <MID> middle_tokens <EOM>

or the reordered SPM variant (Suffix-Prefix-Middle), which some models mix in during training so the choice of order doesn't matter at inference time:

<SUF> suffix_tokens <PRE> prefix_tokens <MID> middle_tokens <EOM>

Either way, the model still generates autoregressively — one token at a time, left to right, only ever appending after <MID>, even though the tokens it is completing land visually in the middle of your file once the special tokens are stripped back out.

Each middle token is drawn from a probability distribution over candidates using temperature-scaled softmax:

p_i = exp(z_i / T) / Σ_j exp(z_j / T)

where z_i are the model's raw logits for candidate i. Low T (→0) collapses the distribution onto the single highest-logit token (greedy decoding); high T flattens it, giving lower-probability — sometimes wrong — candidates a real chance of being sampled. That's the mechanism behind both confident autocomplete and hallucinated code: same formula, different T.

Snippet buttons — three real short functions, each with a missing middle chunk (the "hole") and a small candidate distribution per token position.
PSM / SPM — changes the order the prefix and suffix are fed into the model's input sequence (bottom row); the visible source code (top row) never reorders.
Temperature slider — rescales the softmax before each token is sampled.
Play / Step / Reset — advance the autoregressive generation one middle token at a time and watch perplexity and match-rate update live.

SOURCE CODE (visual order)

MODEL INPUT SEQUENCE (PSM)

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.