Significance Levels
Hypothesis Testing
Setting the Bar for Evidence
The significance level α is the probability threshold for rejecting the null hypothesis, set before data collection. It directly controls the Type I error rate and shapes the stringency of your statistical test.
- Medical Research — Using α = 0.01 or stricter for high-stakes drug approvals
- Social Science — Balancing discovery rigor against exploratory flexibility
- Quality Control — Setting acceptance criteria for incoming materials
Alpha is a decision, not a discovery — choose it wisely before you see the data.
The significance level (α) is the probability threshold below which we consider evidence against H₀ sufficient to reject it. It is set before the test, not after.
What α = 0.05 Actually Means
Demonstrating the False Positive Rate
Conventional Significance Levels by Field
| Field | Common α | Reason |
|---|---|---|
| Social science | 0.05 | Fisher's historical convention |
| Medicine (clinical) | 0.05 or 0.01 | Patient safety concerns |
| Psychology | 0.05 | Replication crisis driving to 0.005 |
| Genetics (GWAS) | 5×10⁻⁸ | Millions of simultaneous tests |
| Particle physics | 0.0000003 (5σ) | Extraordinary claims need extraordinary evidence |
| Quality control | Varies | Depends on cost of errors |
Choosing α: A Decision Framework
The Multiple Testing Problem
Critical Values at Common Significance Levels
# For one-sample z-test (large n)
print("Critical values for z-test:")
print(f"{'α':<8} {'Two-tailed zc':<18} {'One-tailed zc'}")
for alpha in [0.10, 0.05, 0.025, 0.01, 0.005, 0.001]:
zc_two = stats.norm.ppf(1 - alpha/2)
zc_one = stats.norm.ppf(1 - alpha)
print(f"{alpha:<8.3f} ±{zc_two:<17.4f} {zc_one:.4f}")