Central Limit Theorem — The Most Important Theorem in Statistics
Foundations of Statistics
The Theorem That Made Statistics Possible
The CLT explains why the normal distribution appears universally in nature and measurement, justifying nearly all parametric statistical methods. It guarantees that sample means become normal regardless of the underlying distribution.
- Scientific Research — Justifies using t-tests and confidence intervals for means
- Machine Learning — Enables statistical guarantees for ensemble methods and bagging
- Quality Engineering — Underpins statistical process control and Six Sigma
The CLT is the reason statistics works in practice.
The Central Limit Theorem (CLT) is arguably the single most important result in probability and statistics. It explains why the normal distribution appears universally in nature, measurement, and inference.
The Theorem
In words: the standardized sample mean converges in distribution to the standard normal, regardless of the shape of the population distribution.
The CLT Approximation
For large , the CLT gives the approximation:
The variance of the sample mean is — it decreases with sample size, which is why larger samples give more precise estimates.
Rate of Convergence: The Berry–Esseen Theorem
This bound tells us:
- Convergence is not instant — the error is
- For , the maximum error is roughly (if )
- Skewed or heavy-tailed distributions converge more slowly
Minimum Sample Size Guidelines
The CLT for Proportions
A special and widely used case applies to binary (Bernoulli) data:
This is the basis of confidence intervals for proportions and z-tests for proportions.
The CLT for Sums
The standard deviation of grows as — this is why measurement precision improves with , not .
Why the CLT Fails Without Finite Variance
Worked Example: Dice Rolls
Consider rolling a fair die times. Each roll has and . The CLT predicts:
For rolls:
Despite the uniform (non-normal) population, the CLT approximation is excellent for .