Effect Size — Cohen's d, r, η², ω² and When They Matter

Effect Size

Effect size quantifies the magnitude of an effect, independent of sample size. A statistically significant result without effect size is incomplete.

Cohen's d (Mean Difference)

$d = \frac{\bar{x}_1 - \bar{x}_2}{s_p}, \quad s_p = \sqrt{\frac{(n_1-1)s_1^2 + (n_2-1)s_2^2}{n_1+n_2-2}}$

import numpy as np
from scipy import stats
import matplotlib.pyplot as plt

# Cohen's d guidelines
def cohens_d(group1, group2):
    n1, n2 = len(group1), len(group2)
    s1, s2 = group1.std(ddof=1), group2.std(ddof=1)
    sp = np.sqrt(((n1-1)*s1**2 + (n2-1)*s2**2) / (n1+n2-2))
    d = (group1.mean() - group2.mean()) / sp
    size = 'negligible' if abs(d)<0.2 else 'small' if abs(d)<0.5 else 'medium' if abs(d)<0.8 else 'large'
    print(f"Cohen's d = {d:.4f} ({size} effect)")
    return d

np.random.seed(42)
group_a = np.random.normal(75, 10, 50)
group_b = np.random.normal(80, 10, 50)
d = cohens_d(group_a, group_b)

# Overlap visualization
fig, ax = plt.subplots(figsize=(10, 5))
x = np.linspace(40, 120, 500)
ax.plot(x, stats.norm.pdf(x, group_a.mean(), group_a.std()), 'b-', linewidth=2, label=f'Group A (μ={group_a.mean():.1f})')
ax.plot(x, stats.norm.pdf(x, group_b.mean(), group_b.std()), 'r-', linewidth=2, label=f'Group B (μ={group_b.mean():.1f})')
ax.fill_between(x, stats.norm.pdf(x, group_a.mean(), group_a.std()), alpha=0.3, color='blue')
ax.fill_between(x, stats.norm.pdf(x, group_b.mean(), group_b.std()), alpha=0.3, color='red')
ax.set_title(f"Cohen's d = {d:.3f} (medium effect)
Overlap illustrates practical difference")
ax.legend()
plt.tight_layout()
plt.savefig('effect_size_overlap.png', dpi=150)
plt.show()

# ==========================================
# Other effect sizes
# ==========================================

# Pearson r (for correlation)
r_val = 0.40
print(f"
Pearson r = {r_val} → {'small' if abs(r_val)<0.3 else 'medium' if abs(r_val)<0.5 else 'large'} effect")

# Eta-squared (ANOVA: proportion of variance explained)
from scipy.stats import f_oneway
g1, g2, g3 = np.random.normal(50,5,30), np.random.normal(55,5,30), np.random.normal(60,5,30)
F, p = f_oneway(g1, g2, g3)
all_data = np.concatenate([g1, g2, g3])
grand_mean = all_data.mean()
ss_between = sum(len(g)*(g.mean()-grand_mean)**2 for g in [g1,g2,g3])
ss_total = sum((x - grand_mean)**2 for x in all_data)
eta_sq = ss_between / ss_total
omega_sq = (ss_between - (3-1)*all_data.var()) / (ss_total + all_data.var())  # approx
print(f"
ANOVA effect sizes:")
print(f"η² (eta-squared) = {eta_sq:.4f} → {'small' if eta_sq<0.06 else 'medium' if eta_sq<0.14 else 'large'}")
print(f"ω² (omega-squared, less biased) ≈ {max(omega_sq, 0):.4f}")

Effect Size Reference Table

Key Takeaways

1. Always report effect size alongside p-values — p alone is incomplete
2. Cohen's d is the go-to for t-test comparisons
3. Eta-squared (η²) tells you % of variance explained in ANOVA
4. Small effects can matter in large populations (e.g., 0.2 SD improvement in public health)
5. Context matters — what's "small" varies by field