Multiple Comparisons: Bonferroni, Holm, FDR
Every hypothesis test carries a false-positive risk. Run twenty independent tests at α = 0.05 and you expect one "significant" result by chance alone. This is the multiple-comparisons problem, and ignoring it is one of the most common ways to publish a false finding.
Corrections fall into two families: those that control the family-wise error rate (FWER) — the chance of any false positive — and those that control the false discovery rate (FDR) — the expected proportion of false positives among rejections.
Methods compared
| Method | Controls | Use when |
|---|---|---|
| Bonferroni | FWER | Few tests; simplicity |
| Holm | FWER | Always ≥ Bonferroni power |
| Benjamini–Hochberg | FDR | Many tests; discovery |
Bonferroni and why Holm beats it
Bonferroni divides α by the number of tests — simple but conservative. Holm's step-down procedure controls the same FWER with uniformly more power, so prefer Holm over plain Bonferroni.
When to use FDR
For exploratory work with hundreds of comparisons (e.g. screening), Benjamini–Hochberg keeps far more power while bounding the false-discovery proportion.
Reporting
State which correction you applied and report adjusted p-values.
Run this analysis on your own data — free, in your browser.
Open MindStat →