Home▸AI & Machine Learning▸2D Pass@k Unbiased Estimator: Hypergeometric Urn, Plug-in Bias and Benchmark Difficulty

🎯 2D Pass@k Unbiased Estimator: Hypergeometric Urn, Plug-in Bias and Benchmark Difficulty

Benchmark grid of problems by samples with the pass@k curve: the unbiased estimator versus the biased plug-in formula, bias as a function of n, and the Beta difficulty model, verified by a log-gamma form, exact binomial expectation, 200 000 urn shuffles and midpoint integration.

AI & Machine Learning2DModerate60 FPS📱 Mobile-adapted🟠 Blender⇄ 3D version
2D Pass@k Unbiased Estimator: Hypergeometric Urn, Plug-in Bias and Benchmark Difficulty ↗ Open standalone
⚙ Under the hood

Benchmark grid of problems by samples with the pass@k curve: the unbiased estimator versus the biased plug-in formula, bias as a function of n, and the Beta difficulty model, verified by a log-gamma form, exact binomial expectation, 200 000 urn shuffles and midpoint integration.

pass@kCode Generation BenchmarksUnbiased EstimatorHypergeometric SamplingBeta DistributionEstimator BiasLLM EvaluationMonte Carlo

2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)