Home▸Statistics▸Multiple Comparisons: p-Hacking vs Bonferroni

Multiple Comparisons: p-Hacking vs Bonferroni

Run many independent hypothesis tests on pure random noise with no real effect and watch uncorrected testing rack up false 'significant' hits while a Bonferroni-corrected threshold on the exact same data keeps the false-positive rate near the intended 5%.

Statistics2DModerate60 FPS

Every tile in this grid is an independent hypothesis test run on freshly generated random noise that has no real underlying effect at all — the null hypothesis is always true. Run tests one at a time or in batches and watch some come back "statistically significant" (p < 0.05) purely by chance. Toggle between an uncorrected threshold, where the family-wise false-positive rate climbs sharply as you test more hypotheses on the same data, and a Bonferroni-corrected threshold (α divided by the number of tests), which keeps the true false-positive rate pinned near the intended 5% on the exact same random data. Run many families at once and watch the empirical rate converge onto the theoretical curves for both methods, side by side on the live chart.

⚙ Under the hood

Run many independent hypothesis tests on pure random noise with no real underlying effect and watch some come back "statistically significant" purely by chance. Toggle between an uncorrected threshold, where the family-wise false-positive rate climbs sharply as you test more hypotheses on the same data (roughly 64% by 20 tests), and a Bonferroni-corrected threshold that divides alpha by the number of tests, holding the true false-positive rate near the intended 5% on the exact same random data. A live chart plots the theoretical and empirical false-positive rate against the number of tests for both methods side by side.

hypothesis testingmultiple comparisonsp-hackingBonferroni correctionfalse positive ratestatistical significancefamily-wise error rateCanvas2D

2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install