Sampling Methods & Distributions

Overview

Every dataset used in machine learning is a sample from some larger data-generating process. Simple random sampling gives every subset equal probability of selection, making it the gold standard for unbiasedness. Stratified sampling divides the population into subgroups (strata) and samples within each, guaranteeing representation and reducing variance when strata differ. Cluster sampling selects groups and surveys everyone in them, dramatically reducing cost for geographically dispersed populations. Systematic sampling picks every k-th individual after a random start — simple but vulnerable to periodicity. The sampling distribution of a statistic describes how it varies across all possible samples, and the standard error () quantifies that variability. The Central Limit Theorem guarantees that sample means are approximately normal for large , regardless of the population distribution.

Key Concepts

Sampling Methods Comparison

Method	How It Works	Best For	Key Advantage	Key Disadvantage
Simple Random	Equal chance for every subset	Homogeneous populations	Unbiased, easy to analyze	Requires complete sampling frame
Stratified	Sample within each known subgroup	Heterogeneous populations	Lower variance than SRS	Requires prior knowledge of strata
Cluster	Sample clusters, survey everyone	Geographically dispersed	Cheaper than SRS	Higher variance due to ICC
Systematic	Every k-th after random start	Ordered lists	Simple to implement	Vulnerable to periodicity

Sampling Bias Types

Type	Description	Example
Selection bias	Sampling method excludes groups	Voluntary response surveys
Non-response bias	Selected individuals decline	Phone surveys missing workers
Survivorship bias	Only "surviving" cases observed	Studying successful companies only
Undercoverage bias	Some members have zero selection chance	Online-only surveys
Convenience sampling	Easiest-to-reach individuals	Surveying friends and family

Quick Example

Key Takeaways

Deep Dive

For detailed explanations, worked examples, and Python implementations, explore the dedicated statistics lessons:

Population and Sample

Population vs Sample — Parameters vs. statistics, sampling frames, and the goal of statistical inference

Data Collection

Data Collection Methods — Surveys, experiments, observational studies, and their trade-offs

Sampling Techniques

Sampling Techniques — SRS, stratified, cluster, and systematic sampling with formulas, examples, and allocation strategies

Bias and Errors

Sampling Bias and Errors — Selection bias, non-response bias, survivorship bias, famous polling failures, and mitigation strategies

Sampling Methods & Distributions