Conditional Probability
Probability Theory
When New Information Changes Everything You Thought You Knew
Conditional probability answers: "Given that B happened, what is the probability of A?" It restricts the sample space and recalculates within that reduced world.
- Sample space reduction — Condition on B and ignore everything outside it
- Bayesian updating — Use new evidence to revise your beliefs systematically
- Probability trees — Visualize conditional paths through multi-stage experiments
- Independence test — A and B are independent if P(A given B) equals P(A)
Conditional probability is the engine of learning from data. It is how evidence changes beliefs.
What is Conditional Probability?
Definition
The conditional probability of event given that event has occurred is defined as:
This formula restricts the sample space to and recalculates probabilities within that reduced space.
Formal Properties
The Multiplication Rule
Law of Total Probability
Conditional Independence
The Base Rate Fallacy
Worked Example: Two Dice
What is ?
The sample space is restricted to outcomes where the first die is 3: — six equally likely outcomes.
Only gives a sum of 7. Therefore:
This equals , showing that the first die and the sum are independent (when the sum is 7).
Conditional Probability in Machine Learning
| ML Application | Conditional Prob Usage | Why |
|---|---|---|
| Classification | P(class | features) | Core of supervised learning |
| Naive Bayes | P(feature_i | class) | Text classification |
| Bayesian networks | P(X | parents(X)) | Causal inference |
| Weather prediction | P(rain | humidity, pressure) | Probabilistic forecasting |
import numpy as np
from sklearn.naive_bayes import GaussianNB
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
# Conditional probability IS what classifiers learn
digits = load_digits()
X_train, X_test, y_train, y_test = train_test_split(digits.data, digits.target, test_size=0.3)
model = GaussianNB()
model.fit(X_train, y_train)
# model.predict_proba gives P(class | features)
sample = X_test[:1]
proba = model.predict_proba(sample)[0]
print(f"Conditional probability distribution P(class | features):")
for cls, p in enumerate(proba):
bar = '█' * int(p * 50)
print(f" Digit {cls}: {p:.4f} {bar}")
print(f"\nPredicted class: {model.predict(sample)[0]} (highest conditional probability)")