Nonparametric Methods

ℹ️ Why It Matters

When data violates the assumptions of parametric tests (normality, equal variances), nonparametric methods provide robust alternatives. They make no distributional assumptions, working with ranks or permutations instead of raw values. This makes them ideal for skewed data, ordinal scales, small samples, and outliers. In practice, nonparametric methods are often the safer choice when you're unsure about your data's distribution.

Overview

Nonparametric methods replace distributional assumptions with rank-based or permutation-based approaches. The Mann-Whitney U test is the nonparametric alternative to the two-sample t-test, comparing distributions using ranks. The Wilcoxon signed-rank test replaces the paired t-test for matched data. The Kruskal-Wallis test extends to multiple groups (nonparametric ANOVA). The Friedman test handles repeated measures. Permutation tests build exact null distributions by shuffling labels, providing p-values for any test statistic. Bootstrap methods approximate sampling distributions without assumptions. These tests are slightly less powerful than parametric tests when assumptions hold, but far more robust when they don't.

Key Concepts

Nonparametric Test Equivalents

Nonparametric Test	Parametric Equivalent	Use Case	What It Tests
Mann-Whitney U	Two-sample t-test	Independent samples	Distributions differ
Wilcoxon signed-rank	Paired t-test	Paired samples	Median difference ≠ 0
Kruskal-Wallis	One-way ANOVA	Multiple independent groups	At least one group differs
Friedman	Repeated-measures ANOVA	Multiple related groups	Treatment effects
Permutation test	Any parametric test	Any test statistic	Any null hypothesis

Mann-Whitney U Statistic

U = n_1 n_2 + \frac{n_1(n_1+1)}{2} - R_1

Here,

$R_1$ =Sum of ranks for group 1
$n_1, n_2$ =Sample sizes for each group

Wilcoxon Signed-Rank Statistic

W = \sum_{i: d_i > 0} R_i

Here,

$d_i$ =Difference for pair i
$R_i$ =Rank of |d_i| among all |d_j|

Kruskal-Wallis H Statistic

H = \frac{12}{N(N+1)} \sum_{i=1}^{k} \frac{R_i^2}{n_i} - 3(N+1)

Here,

$R_i$ =Sum of ranks for group i
$N$ =Total sample size
$k$ =Number of groups

Permutation Test P-Value

p = \frac{\text{count of permuted statistics} \geq \text{observed}}{\text{total permutations}}

Here,

$n_{perm}$ =Number of permutations (use all or large B)

When to Use Each Test

Data Characteristic	Recommended Test
Skewed, 2 independent groups	Mann-Whitney U
Skewed, paired data	Wilcoxon signed-rank
Skewed, 3+ independent groups	Kruskal-Wallis
Ordinal data	Any rank-based test
Very small sample ( $n < 10$ )	Permutation test
Complex test statistic	Bootstrap

Quick Example

📝Choosing the Right Test

Data is heavily skewed (exponential distribution). Two groups need comparison.

A two-sample t-test assumes normality, which is violated here. Use the Mann-Whitney U test instead — it compares distributions using ranks and is robust to skewness and outliers. The null hypothesis is that the two groups have the same distribution, not that their means are equal.

📝Permutation Test

Observed difference in means: 5.3. After 10,000 label shuffles, 237 permutations produce a difference ≥ 5.3.

$p = 237/10000 = 0.0237$ . Reject $H_0$ at $\alpha = 0.05$ . The permutation test makes no distributional assumptions and provides an exact p-value.

Key Takeaways

📋Summary: Nonparametric Methods

When Normality Fails: Use nonparametric tests when data is skewed, ordinal, or has heavy tails.
Rank-Based Tests: Mann-Whitney U (independent), Wilcoxon signed-rank (paired), Kruskal-Wallis (multiple groups).
Bootstrap CIs: Resample with replacement to approximate sampling distributions. Works for any statistic.
Permutation Tests: Build exact null distributions by shuffling labels. No distributional assumptions. Valid for small samples.
Trade-off: Nonparametric tests are more robust but slightly less powerful than parametric tests when assumptions hold.
Interpretation: These tests compare medians or distributions, not means. The research question is subtly different from parametric tests.
Small Samples: Permutation tests are especially valuable when $n$ is too small for CLT-based approximations.

Deep Dive

For detailed explanations, worked examples, and Python implementations, explore the dedicated statistics lessons:

Permutation Tests

Permutation Tests — Exact p-values by shuffling labels, no distributional assumptions, and when to use

Paired Tests

Wilcoxon Signed-Rank Test — Nonparametric paired test for medians with full derivation

Independent Group Tests

Mann-Whitney U Test — Nonparametric two-sample test using ranks with examples

Multiple Group Tests

Kruskal-Wallis Test — Nonparametric one-way ANOVA using ranks

Repeated Measures

Friedman Test — Nonparametric repeated-measures ANOVA for related groups

Nonparametric Methods