Hypothesis Testing Explained
Formalize questions as hypotheses, choose a test, and balance false positives and false negatives.
📚 Fundamentals
- Null vs Alternative: baseline vs effect
- p-value: extremeness under the null
- Power: probability of detecting true effects
❓ Frequently Asked Questions
1) Is p-value the probability the null is true?
No; it's the probability of data as extreme under the null.
No; it's the probability of data as extreme under the null.
2) Multiple testing?
Control FDR or adjust with Bonferroni/Holm.
Control FDR or adjust with Bonferroni/Holm.
3) Effect size?
Report alongside p-values for practical significance.
Report alongside p-values for practical significance.
4) Confidence intervals?
Give a range of plausible values; complements tests.
Give a range of plausible values; complements tests.
5) Bayesian alternatives?
Credible intervals and Bayes factors.
Credible intervals and Bayes factors.