Bootstrap Confidence Intervals — Resampling-Based Inference
Foundations of Statistics
When Theory Meets Reality
Bootstrap methods use resampling to construct confidence intervals without distributional assumptions, making them applicable to virtually any statistic. They provide reliable inference even when traditional formulas break down.
- Finance — Constructing confidence intervals for risk measures like VaR
- Ecology — Estimating uncertainty for biodiversity indices
- Machine Learning — Quantifying variability in model performance metrics
The bootstrap lets the data speak for itself.
What Is Bootstrap Confidence Intervals?
Core Concept
Theoretical Foundation
Key insight: We replace the unknown population with the known empirical , then use the computer to simulate what we cannot compute analytically.
Three Bootstrap CI Methods
Worked Example: Median Bootstrap CI
Given data (), compute a 95% BCA bootstrap CI for the population median.
Step 1: Compute the sample median: .
Step 2: Draw bootstrap samples, compute the median of each.
Step 3: Apply the BCA correction:
- is estimated via the jackknife:
where is the median computed on the sample with the -th observation deleted.
Step 4: Adjusted quantiles give the BCA interval.
Python Implementation: Three Bootstrap Methods
import numpy as np
from scipy import stats
np.random.seed(42)
data = np.array([2.1, 3.5, 4.2, 1.8, 5.6, 3.9, 2.7, 4.8, 3.2, 6.1, 2.9, 5.0])
theta_hat = np.median(data)
B = 10000
# Generate bootstrap samples and compute medians
boot_medians = np.array([np.median(np.random.choice(data, size=len(data), replace=True))
for _ in range(B)])
# Percentile CI
ci_percentile = np.percentile(boot_medians, [2.5, 97.5])
# BCA CI (using jackknife for acceleration)
n = len(data)
jack_medians = np.array([np.median(np.delete(data, i)) for i in range(n)])
jack_mean = np.mean(jack_medians)
a_hat = np.sum((jack_mean - jack_medians)**3) / (6 * np.sum((jack_mean - jack_medians)**2)**1.5)
z0 = stats.norm.ppf(np.mean(boot_medians < theta_hat))
z_alpha = stats.norm.ppf([0.025, 0.975])
alpha1 = stats.norm.cdf(z0 + (z0 + z_alpha) / (1 - a_hat * (z0 + z_alpha)))
ci_bca = np.percentile(boot_medians, alpha1 * 100)
# Studentized CI
def se_median(x, B_boot=1000):
boots = np.array([np.median(np.random.choice(x, len(x), replace=True)) for _ in range(B_boot)])
return np.std(boots)
se_hat = se_median(data)
boot_t = (boot_medians - theta_hat) / se_hat
ci_studentized = theta_hat - np.percentile(boot_t, [97.5, 2.5]) * se_hat
print(f"Sample median: {theta_hat:.3f}")
print(f"Percentile CI: [{ci_percentile[0]:.3f}, {ci_percentile[1]:.3f}]")
print(f"BCA CI: [{ci_bca[0]:.3f}, {ci_bca[1]:.3f}]")
print(f"Studentized CI:[{ci_studentized[0]:.3f}, {ci_studentized[1]:.3f}]")