Statistics 101: Mean, Median, Variance and Distributions

Why Statistics Matters for Data Science

Statistics is the mathematical foundation of machine learning. Every ML algorithm is essentially a statistical model optimizing some objective function. Without understanding statistics, you cannot truly understand why models work or fail.

The Statistics → ML Pipeline

1. Measures of Central Tendency

Central tendency describes the "center" or "typical value" of a dataset.

Mean (Arithmetic Average)

Formula:

\bar{x} = \frac{1}{n} \sum_{i=1}^{n} x_i = \frac{x_1 + x_2 + \cdots + x_n}{n}

Median (Middle Value)

The median is the value separating the higher half from the lower half of a data sample.

Formula:

\text{Median} = \begin{cases} x_{(n+1)/2} & \text{if } n \text{ is odd} \\ \frac{x_{n/2} + x_{n/2+1}}{2} & \text{if } n \text{ is even} \end{cases}

Mode (Most Frequent)

2. Measures of Spread (Dispersion)

Variance and Standard Deviation

Population Variance:

\sigma^2 = \frac{1}{N} \sum_{i=1}^{N} (x_i - \mu)^2

Sample Variance (Bessel's Correction):

s^2 = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x})^2

Standard Deviation:

\sigma = \sqrt{\sigma^2}, \quad s = \sqrt{s^2}

σ² = (16 + 4 + 0 + 4 + 16 + 0) / 6 = 40/6 = 6.67

σ = √6.67 = 2.58

Negative deviationPositive deviationOn the mean

3. The Normal Distribution (Gaussian)

The most important probability distribution in statistics and ML.

Probability Density Function (PDF):

f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2}

Where:

$\mu$ = mean (location parameter)
$\sigma$ = standard deviation (scale parameter)
$\sigma^2$ = variance

Standard Normal Distribution (Z-Score)

Z-Score Transformation:

Z = \frac{X - \mu}{\sigma}

This transforms any normal distribution to the standard normal with $\mu = 0$ and $\sigma = 1$ .

4. Central Limit Theorem (CLT)

The most important theorem in statistics — explains why the normal distribution appears everywhere.

Central Limit Theorem:

Given a population with mean $\mu$ and standard deviation $\sigma$ , the sampling distribution of the sample mean $\bar{X}$ approaches a normal distribution as sample size $n$ increases:

\bar{X} \sim N\left(\mu, \frac{\sigma^2}{n}\right)

Standard Error:

SE = \frac{\sigma}{\sqrt{n}}

Key Insight: As n increases, the sampling distribution becomes more normal

σ̄ = σ/√n → Standard error decreases with √n

Practical Impactn=1SE = σn=4SE = σ/2n=16SE = σ/4n=100SE = σ/10

5. Skewness and Kurtosis

Skewness (Asymmetry)

Mean > Median → Right Skewed | Mean < Median → Left Skewed

Kurtosis (Tail Weight)

Fisher's Kurtosis:

\gamma_2 = \frac{\mu_4}{\sigma^4} - 3 = \frac{E[(X-\mu)^4]}{(E[(X-\mu)^2])^2} - 3

Mesokurtic (γ₂ = 0): Normal distribution
Leptokurtic (γ₂ > 0): Heavy tails, sharp peak
Platykurtic (γ₂ < 0): Light tails, flat peak

Higher kurtosis = more outliers, fatter tails, sharper peak

6. Covariance and Correlation

Covariance

Covariance:

\text{Cov}(X, Y) = \sigma_{XY} = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})

Cov > 0: Variables move together (positive)
Cov < 0: Variables move opposite (negative)
Cov = 0: No linear relationship

Pearson Correlation Coefficient

Pearson's r:

r = \frac{\text{Cov}(X,Y)}{\sigma_X \sigma_Y} = \frac{\sum_{i=1}^{n}(x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_{i=1}^{n}(x_i - \bar{x})^2 \cdot \sum_{i=1}^{n}(y_i - \bar{y})^2}}

r \in [-1, +1]

Note: Correlation measures LINEAR relationship only. Non-linear patterns need other metrics.

7. Confidence Intervals

Confidence Interval for Mean:

\bar{x} \pm z_{\alpha/2} \cdot \frac{s}{\sqrt{n}}

Common Confidence Levels:

Level	z-score	Area in Tails
90%	1.645	5% each side
95%	1.960	2.5% each side
99%	2.576	0.5% each side

We are 95% confident that the true mean lies between 38 and 62

Key Takeaways

Mean, Median, Mode describe central tendency — choose based on data shape
Variance/Standard Deviation quantify spread — foundation of all ML loss functions
Normal Distribution — the bell curve underlies hypothesis testing and CLT
CLT — sample means are normal regardless of population shape (n ≥ 30)
Correlation ≠ Causation — always consider confounding variables
Confidence Intervals — quantify uncertainty in estimates

Next: Probability, Bayes' Theorem and PDF/CDF

Build on these foundations with probability theory and Bayesian inference.

Statistics 101: Mean, Median, Variance and Distributions

Statistics 101: Mean, Median, Variance and Distributions

Why Statistics Matters for Data Science

The Statistics → ML Pipeline

1. Measures of Central Tendency

Mean (Arithmetic Average)

Median (Middle Value)

Mode (Most Frequent)

2. Measures of Spread (Dispersion)

Variance and Standard Deviation

3. The Normal Distribution (Gaussian)

Standard Normal Distribution (Z-Score)

4. Central Limit Theorem (CLT)

5. Skewness and Kurtosis

Skewness (Asymmetry)

Kurtosis (Tail Weight)

6. Covariance and Correlation

Covariance

Pearson Correlation Coefficient

7. Confidence Intervals

Key Takeaways

Next: Probability, Bayes' Theorem and PDF/CDF

Need Expert Data Science Help?