Random Variables & Probability Distributions

What is a Random Variable?

Simple Analogy: Think of a random variable like a score in a game. The game has many possible outcomes (roll of a dice, draw of a card), but the random variable converts each outcome into a number you can work with — like points, dollars, or centimeters.

Real-World Examples:

Coin flip: if heads, if tails
Dice roll: = the number shown on the die (1 through 6)
Height of a person: = height in centimeters (can be any value in a range)
Number of customers: = count of customers arriving in an hour (0, 1, 2, ...)

Discrete vs. Continuous

Examples of discrete variables:

Number of heads in 10 coin flips:
Number of emails received per day:
Customer rating:

Examples of continuous variables:

Temperature: any value in degrees Celsius
Weight: any value in kilograms
Time to complete a task: any value in seconds

Probability Mass Function (PMF)

Probability Density Function (PDF)

Cumulative Distribution Function (CDF)

Bernoulli Distribution

The simplest distribution: a single yes/no trial.

Binomial Distribution

How many successes in independent Bernoulli trials?

Poisson Distribution

Counting rare events over a fixed interval.

Uniform Distribution

Every outcome equally likely over an interval.

Python Implementation

import numpy as np
from scipy import stats
import matplotlib.pyplot as plt

# --- Bernoulli Distribution ---
# Simulate 10000 coin flips with p=0.7
bernoulli_rv = stats.bernoulli(p=0.7)
samples = bernoulli_rv.rvs(size=10000)
print(f"Bernoulli Mean: {samples.mean():.3f}")       # ~0.70
print(f"Bernoulli Var: {samples.var():.3f}")          # ~0.21
print(f"P(X=1): {bernoulli_rv.pmf(1):.3f}")          # 0.700
print(f"P(X<=0): {bernoulli_rv.cdf(0):.3f}")         # 0.300

# --- Binomial Distribution ---
# 10 trials, p=0.3, simulate 10000 experiments
binom_rv = stats.binom(n=10, p=0.3)
samples = binom_rv.rvs(size=10000)
print(f"Binomial Mean: {samples.mean():.3f}")         # ~3.0
print(f"Binomial Var: {samples.var():.3f}")           # ~2.1
print(f"P(X=5): {binom_rv.pmf(5):.4f}")              # ~0.1029
print(f"P(X<=3): {binom_rv.cdf(3):.4f}")             # ~0.6496

# --- Poisson Distribution ---
# Average 4 events per interval
poisson_rv = stats.poisson(mu=4)
samples = poisson_rv.rvs(size=10000)
print(f"Poisson Mean: {samples.mean():.3f}")          # ~4.0
print(f"Poisson Var: {samples.var():.3f}")            # ~4.0
print(f"P(X=6): {poisson_rv.pmf(6):.4f}")            # ~0.1042

# --- Uniform Distribution ---
# Continuous uniform on [0, 1]
uniform_rv = stats.uniform(loc=0, scale=1)
samples = uniform_rv.rvs(size=10000)
print(f"Uniform Mean: {samples.mean():.3f}")          # ~0.50
print(f"Uniform Var: {samples.var():.4f}")            # ~0.0833
print(f"P(0.25<=X<=0.75): {uniform_rv.cdf(0.75) - uniform_rv.cdf(0.25):.3f}")  # 0.500

# --- Visualization ---
fig, axes = plt.subplots(2, 2, figsize=(12, 8))

# Bernoulli
axes[0, 0].bar([0, 1], [0.3, 0.7], color=['steelblue', 'coral'])
axes[0, 0].set_title('Bernoulli(p=0.7)')
axes[0, 0].set_xlabel('x')
axes[0, 0].set_ylabel('P(X=x)')

# Binomial
x_binom = np.arange(0, 11)
axes[0, 1].bar(x_binom, binom_rv.pmf(x_binom), color='steelblue')
axes[0, 1].set_title('Binomial(n=10, p=0.3)')
axes[0, 1].set_xlabel('k')
axes[0, 1].set_ylabel('P(X=k)')

# Poisson
x_poisson = np.arange(0, 12)
axes[1, 0].bar(x_poisson, poisson_rv.pmf(x_poisson), color='coral')
axes[1, 0].set_title('Poisson(λ=4)')
axes[1, 0].set_xlabel('k')
axes[1, 0].set_ylabel('P(X=k)')

# Uniform
x_uniform = np.linspace(-0.2, 1.2, 1000)
axes[1, 1].fill_between(x_uniform, uniform_rv.pdf(x_uniform), alpha=0.3, color='steelblue')
axes[1, 1].plot(x_uniform, uniform_rv.pdf(x_uniform), color='steelblue')
axes[1, 1].set_title('Uniform(0, 1)')
axes[1, 1].set_xlabel('x')
axes[1, 1].set_ylabel('f(x)')

plt.tight_layout()
plt.savefig('distributions.png', dpi=150)
plt.show()

Applications in AI/ML

Loss Functions Derived from Distributions

Many common loss functions in ML are negative log-likelihoods of probability distributions:

Loss Function	Distribution	Use Case
Binary Cross-Entropy	Bernoulli	Binary classification
Categorical Cross-Entropy	Categorical	Multi-class classification
MSE (Mean Squared Error)	Gaussian	Regression
Poisson Loss	Poisson	Count prediction

Sampling and Data Augmentation

Monte Carlo methods: Draw samples from distributions to estimate integrals and expectations
Reparameterization trick: Used in VAEs (Variational Autoencoders) to backpropagate through random sampling
Data augmentation: Add noise sampled from known distributions to training data

Generative Models

Gaussian Mixture Models (GMM): Model data as a mixture of Gaussians
Naive Bayes: Assume features follow specific distributions (Gaussian, Bernoulli, Multinomial)
Normalizing Flows: Transform simple distributions (Uniform, Gaussian) into complex ones

Common Mistakes

Mistake	Why It's Wrong	Correct Approach
Saying for a continuous variable	For continuous RVs, the probability at a single point is always 0	Use intervals:
Treating PDF values as probabilities	is a density, not a probability; it can exceed 1	Probabilities are areas under the curve:
Using PMF for continuous variables	PMFs are only defined for discrete variables	Use PDF for continuous, PMF for discrete
Forgetting or	If these don't hold, it's not a valid distribution	Always verify normalization
Confusing and	is the population mean (parameter); is the sample mean (statistic)	is fixed; varies by sample
Assuming independence when it's not given	Independence is a strong assumption that must be justified	Check the problem statement carefully
Using Binomial when trials are not independent	Binomial requires independent trials	Use Hypergeometric for sampling without replacement

Interview Questions

Practice Problems

Quick Reference

Cross-References

Previous: Probability Fundamentals — Sample spaces, events, conditional probability, Bayes' theorem
Next: Expectation and Variance — Mean, variance, standard deviation, higher moments
Related: Linear Algebra — Vector spaces, matrix operations
Related: Information Theory — Entropy, KL divergence, mutual information
Applied: Loss Functions in ML — Cross-entropy, MSE, and their distributional origins
Applied: Bayesian Methods — Prior distributions, posterior inference, MCMC

Random Variables & Probability Distributions