The Multiple Testing Problem
Hypothesis Testing
When Testing More Means Finding More Wrong
The multiple testing problem causes false discovery rates to explode when running many hypothesis tests simultaneously. Without correction, most "significant" findings in large-scale studies are false positives.
- Genomics — Testing thousands of genes for differential expression in disease studies
- Neuroimaging — Analyzing millions of brain voxels for activation differences
- Quality Control — Inspecting multiple product characteristics simultaneously
The more you test, the more you must correct — or drown in false discoveries.
If you run 20 hypothesis tests at α = 0.05 when all H₀s are true, you expect 1 false positive by chance. Run 1000 tests? Expect 50 false discoveries.