Bernoulli Distribution
Probability Distributions
The Simplest Random Experiment — Success or Failure
The Bernoulli distribution is the atom of probability: a single yes/no outcome. From this simplicity, entire empires of statistical theory are built.
- Coin flips — heads or tails, each with probability
- Medical tests — positive or negative diagnosis
- Quality control — item passes or fails inspection
- Machine learning — binary classification labels
Every complex random variable can be decomposed into Bernoulli trials. Master this, and you master the foundation.
Core Concepts
The Bernoulli distribution is the canonical example of a discrete random variable and the fundamental building block for all counting processes. Every sequence of independent binary trials — coin flips, yes/no decisions, pass/fail outcomes — is modeled as a collection of Bernoulli random variables.
Mean and Variance: Derivation
Higher Moments and the Bernoulli Property
Cumulant and Moment Generating Functions
The cumulant generating function is , and the cumulants are:
The third cumulant (skewness indicator) is positive for and negative for , reflecting the asymmetry of the distribution.
Relationship to Other Distributions
Worked Example: Drug Efficacy Trial
Python Implementation
import numpy as np
from scipy import stats
np.random.seed(42)
# Simulate Bernoulli trials
p = 0.7
n = 10000
samples = np.random.binomial(1, p, size=n)
# Verify mean and variance
print(f"Bernoulli(p={p})")
print(f" Empirical mean: {np.mean(samples):.4f} (theoretical: {p})")
print(f" Empirical variance: {np.var(samples, ddof=0):.4f} (theoretical: {p*(1-p):.4f})")
# Verify X^2 = X property
print(f" E[X^2]: {np.mean(samples**2):.4f} (should equal E[X])")
# Show variance as function of p
p_values = np.linspace(0.01, 0.99, 20)
variances = p_values * (1 - p_values)
print(f"\nMax variance at p=0.5: {0.5 * 0.5:.4f}")
Python Implementation: Bernoulli Process Simulation
import numpy as np
np.random.seed(42)
# Simulate a Bernoulli process: sequence of iid trials
p = 0.4
n_trials = 20
process = np.random.binomial(1, p, size=n_trials)
# Compute inter-arrival times (trials between successes, including the success trial)
success_indices = np.where(process == 1)[0]
inter_arrival = np.diff(np.concatenate([[-1], success_indices]))
print(f"Bernoulli process (p={p}): {process}")
print(f"Success positions: {success_indices}")
print(f"Inter-arrival times: {inter_arrival}")
print(f"Theoretical mean inter-arrival: {1/p:.2f}")
print(f"Empirical mean inter-arrival: {np.mean(inter_arrival):.2f}")
# Number of successes in first 10 vs last 10 trials
print(f"Successes in first 10: {np.sum(process[:10])} (Binomial(10, {p}))")
print(f"Successes in last 10: {np.sum(process[10:])} (Binomial(10, {p}))")