Every study tests the same null hypothesis: "there is nothing unusual here." A small slice of studies (the true anomaly rate) really are looking at a genuine effect and, with typical statistical power, correctly come back significant most of the time. Every other study is testing pure noise — and by the definition of a p-value, a noise study still comes back "significant" (p < α) purely by chance at a rate of exactly α, no matter how many times you run it.
P(false positive per null study) = α
E[false positives] = α · (1 − true rate) · N
E[true positives] = power · true rate · N
apparent rate = published / N ≈ α(1−p) + power·p
When only significant results get written up — the file-drawer effect — every null result that correctly found nothing simply vanishes from view. The reader never sees the denominator, only a stream of "anomaly confirmed" papers. As studies run grows, the false positives (which scale with N) swamp the rare true positives (which scale with p·N), so the published rate and the false discovery rate among published work climb well above the true rate — this is the same multiple-comparisons mechanism behind the reproducibility crisis and p-hacking in real meta-science.
- Significance threshold α — the p-value cutoff. Lower it and fewer noise studies sneak through as "significant," but it never reaches zero.
- True anomaly rate — how often a study is actually looking at something real. In most exploratory fields this is small.
- Publish only significant — toggle off to simulate a registry where every result, null or not, stays visible; watch the published rate collapse back toward the true rate.