Expected Value
Probability Theory
The Long-Run Average — What Happens on Average
Expected value is the weighted average of all possible values a random variable can take, weighted by their probabilities. It is the single number that summarizes a random variable.
- Law of Large Numbers — The average of many trials converges to the expected value
- Linearity of expectation — E[X + Y] equals E[X] plus E[Y], always
- Decision theory — Choose the action with the highest expected payoff
- Fair game — A game is fair if its expected value is zero
Expected value is the most important concept in probability. It tells you what to expect on average.
What is Expected Value?
Definition
Expected value is the weighted average of all possible values a random variable can take, weighted by their probabilities. It is the long-run average value over infinitely many repetitions of the experiment.
import numpy as np
import matplotlib.pyplot as plt
# Fair die
values = np.array([1, 2, 3, 4, 5, 6])
probs = np.array([1/6, 1/6, 1/6, 1/6, 1/6, 1/6])
expected = np.sum(values * probs)
print(f"E[Die] = {' + '.join(f'{v}×(1/6)' for v in values)}")
print(f" = {expected:.4f}")
print(f" = 21/6 = 3.5")
Properties of Expected Value
# Verify linearity
np.random.seed(42)
n_sim = 100000
X = np.random.choice([1, 2, 3, 4, 5, 6], n_sim)
Y = np.random.choice([1, 2, 3, 4, 5, 6], n_sim)
print(f"E[X] ≈ {X.mean():.4f} (theoretical: 3.5)")
print(f"E[Y] ≈ {Y.mean():.4f} (theoretical: 3.5)")
print(f"E[X+Y] ≈ {(X+Y).mean():.4f} (theoretical: 7.0)")
print(f"E[X]+E[Y] ≈ {X.mean()+Y.mean():.4f}")
Expected Value of Functions
# E[X²] for a fair die
squared_values = values**2
e_x_squared = np.sum(squared_values * probs)
variance = e_x_squared - expected**2
print(f"E[X²] = {' + '.join(f'{v}²×(1/6)' for v in values)}")
print(f" = {e_x_squared:.4f}")
print(f"\nVar(X) = E[X²] - (E[X])² = {e_x_squared:.4f} - {expected**2:.4f} = {variance:.4f}")
Expected Value in Decision Making
# Expected Value of a Lottery Ticket
cost = 10
prizes = np.array([1000, 100, 10, 0])
probabilities = np.array([0.001, 0.01, 0.1, 0.889])
expected_prize = np.sum(prizes * probabilities)
net_ev = expected_prize - cost
print("Lottery Expected Value:")
for p, prob in zip(prizes, probabilities):
print(f" Prize ${p:>5}: P = {prob:.3f}")
print(f"\n E[Prize] = ${expected_prize:.2f}")
print(f" Cost = ${cost:.2f}")
print(f" Net EV = ${net_ev:.2f}")
print(f" -> {'Unfavorable' if net_ev < 0 else 'Favorable'} bet!")
Conditional Expected Value
# E[X | condition]
np.random.seed(42)
scores = np.random.normal(75, 12, 1000)
overall_mean = scores.mean()
high_scores = scores[scores > 80]
low_scores = scores[scores <= 80]
print(f"E[Score] = {overall_mean:.2f}")
print(f"E[Score | Score > 80] = {high_scores.mean():.2f}")
print(f"E[Score | Score ≤ 80] = {low_scores.mean():.2f}")
Expected Value in Machine Learning
| ML Application | Expected Value Usage | Why |
|---|---|---|
| Loss functions | E[loss] = expected risk | Minimize expected loss |
| Reinforcement learning | E[reward] = value function | Maximize expected return |
| Thompson sampling | E[outcome | arm] | Multi-armed bandits |
| Value at Risk | E[loss | threshold] | Financial ML |
import numpy as np
# Expected value = what you expect on average
# E[X] = Σ x × P(x)
# Example: portfolio expected return
returns = np.array([0.10, 0.05, -0.03, 0.08])
probabilities = np.array([0.3, 0.4, 0.1, 0.2])
expected_return = np.sum(returns * probabilities)
print(f"Returns: {returns}")
print(f"Probabilities: {probabilities}")
print(f"Expected return: {expected_return:.4f} ({expected_return*100:.2f}%)")
# Reinforcement learning: Q-value = expected cumulative reward
# Q(s,a) = E[G_t | s_t=s, a_t=a]
print(f"\nIn RL: Q(s,a) = E[total reward | state s, action a]")
print(f"This is just the expected value of future rewards!")