Effect Size
Hypothesis Testing
How Big Is the Real Effect?
Effect size measures the magnitude of an effect independent of sample size, answering the practical significance question that p-values cannot. It transforms statistical significance into meaningful conclusions.
- Meta-Analysis — Combining results across studies using standardized effect measures
- Education Research — Evaluating whether interventions produce practically important gains
- Business Decisions — Assessing whether improvements justify implementation costs
Statistical significance tells you if an effect exists; effect size tells you if it matters.
Cohen's d (Mean Difference)
Python Implementation
Overlap Visualization
# Overlap visualization
fig, ax = plt.subplots(figsize=(10, 5))
x = np.linspace(40, 120, 500)
ax.plot(x, stats.norm.pdf(x, group_a.mean(), group_a.std()), 'b-', linewidth=2, label=f'Group A (μ={group_a.mean():.1f})')
ax.plot(x, stats.norm.pdf(x, group_b.mean(), group_b.std()), 'r-', linewidth=2, label=f'Group B (μ={group_b.mean():.1f})')
ax.fill_between(x, stats.norm.pdf(x, group_a.mean(), group_a.std()), alpha=0.3, color='blue')
ax.fill_between(x, stats.norm.pdf(x, group_b.mean(), group_b.std()), alpha=0.3, color='red')
ax.set_title(f"Cohen's d = {d:.3f} (medium effect)\nOverlap illustrates practical difference")
ax.legend()
plt.tight_layout()
plt.savefig('effect_size_overlap.png', dpi=150)
plt.show()
Other Effect Size Measures
Effect Size Reference Table
| Measure | Small | Medium | Large | Used For |
|---|---|---|---|---|
| Cohen's d | 0.2 | 0.5 | 0.8 | Mean differences (t-test) |
| Pearson r | 0.1 | 0.3 | 0.5 | Correlations |
| R² | 0.01 | 0.09 | 0.25 | Regression |
| η² | 0.01 | 0.06 | 0.14 | ANOVA |
| Cramér's V | 0.1 | 0.3 | 0.5 | Chi-square |
| Cohen's f | 0.1 | 0.25 | 0.40 | ANOVA (alternative) |