⚖️ 2D LLM Judge Position Bias: Probit Choice Model, Swap Consistency and Order-Averaged Debiasing
Score densities for the same answer pair shown in both orders, the shaded chance of picking the first slot, accuracy against quality gap for every policy and a live tally compared with closed forms; swapping and averaging cancels the bias.
⚙ Under the hood
Score densities for the same answer pair shown in both orders, the shaded chance of picking the first slot, accuracy against quality gap for every policy and a live tally compared with closed forms; swapping and averaging cancels the bias.
LLM-as-a-JudgePosition BiasEvaluationProbit ModelSwap ConsistencyDebiasingMonte CarloBenchmarks
2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install