Home▸AI & Machine Learning▸3D Pass@k Unbiased Estimator: Hypergeometric Urn, Plug-in Bias and Benchmark Difficulty

🎯 3D Pass@k Unbiased Estimator: Hypergeometric Urn, Plug-in Bias and Benchmark Difficulty

A glass urn built in Blender holds n sampled programs, c of them correct; draw k without replacement and compare the unbiased pass@k estimate 1 - C(n-c,k)/C(n,k) with the naive plug-in value and the Beta-mixture truth, checked by exact binomial expectation, urn shuffles and numerical integration.

AI & Machine Learning3DModerate60 FPS📱 Mobile-adapted🟠 Blender⇄ 2D version
3D Pass@k Unbiased Estimator: Hypergeometric Urn, Plug-in Bias and Benchmark Difficulty ↗ Open standalone
⚙ Under the hood

A glass urn built in Blender holds n sampled programs, c of them correct; draw k without replacement and compare the unbiased pass@k estimate 1 - C(n-c,k)/C(n,k) with the naive plug-in value and the Beta-mixture truth, checked by exact binomial expectation, urn shuffles and numerical integration.

pass@kCode Generation BenchmarksUnbiased EstimatorHypergeometric SamplingBeta DistributionEstimator BiasLLM EvaluationMonte Carlo

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)