Task passed
Task failed
Bootstrap Δ distribution
âš Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.
Two tool-using LLM agents each run a seeded battery of pass/fail tasks, and the question this simulator answers is the one every real evaluation harness has to answer: is the difference in their success rates real, or could it just as easily be noise? It renders both agents' task outcomes as a 3D grid of pass/fail blocks, runs a permutation test that reshuffles which trials "belong" to which agent thousands of times to build an empirical null distribution, and a bootstrap resample to produce a 95% confidence interval on the success-rate gap — the same resampling methodology used to validate benchmark claims before trusting them.