Measure Theory
Core Definitions
Key Formulas
Important Theorems
Worked Examples
Practice Problems
Common Mistakes
| Mistake | Correct Approach |
|---|---|
| Assuming pointwise convergence implies convergence of integrals | Need dominated convergence (DCT) or monotone convergence (MCT) |
| Confusing "almost everywhere" with "everywhere" | Sets of measure zero can be ignored in Lebesgue integration |
| Forgetting Fubini's theorem requires integrability | Check before swapping integration order |
| Assuming the Riemann and Lebesgue integrals always agree | They agree for Riemann-integrable functions, but Lebesgue handles more cases |
| Confusing σ-algebra with topology | σ-algebras are closed under complement and countable union; topologies are closed under arbitrary union and finite intersection |
| Assuming every subset is measurable | There exist non-measurable sets (Vitali sets) requiring the Axiom of Choice |
| Forgetting that measures are only countably additive | Finite additivity is weaker; countable additivity enables limits |
Connections to Machine Learning
Exam/Interview Questions
Q1: State the three convergence theorems and explain when each applies.
Answer: (1) Monotone Convergence Theorem: If with , then . Requires monotone increase. (2) Fatou's Lemma: For non-negative , . Always holds but gives inequality. (3) Dominated Convergence Theorem: If a.e. and with integrable, then . Requires an integrable bound. MCT is used for increasing sequences, DCT for general convergence with domination, and Fatou for lower bounds.
Q2: Why is the Lebesgue integral preferred over the Riemann integral in probability theory?
Answer: The Lebesgue integral handles: (1) highly discontinuous functions (like indicator functions of rationals), (2) infinite-dimensional spaces (function spaces, sequence spaces), (3) general measures (not just length on ), and (4) convergence theorems that justify limit-interchange operations essential in probability (law of large numbers, central limit theorem). Riemann integration cannot handle the Dirichlet function, and cannot be extended to general measure spaces.
Q3: What is a σ-algebra and why do we need it?
Answer: A σ-algebra on is a collection of subsets closed under complement and countable union. We need it because: (1) Not all subsets of are Lebesgue measurable (Vitali sets). (2) The complement axiom ensures . (3) Countable union closure enables countable additivity: for disjoint , which is essential for taking limits of events. The Borel σ-algebra is generated by open sets.
Q4: Give an example where the Dominated Convergence Theorem does not apply and the conclusion fails.
Answer: Let on . Then a.e., but . The DCT fails because there is no integrable dominating function: , and any dominating would need on for all , making . This illustrates the necessity of the domination condition.
Q5: Explain the Radon-Nikodym theorem and its interpretation as a likelihood ratio.
Answer: If (Q is absolutely continuous w.r.t. P), then exists such that . In probability: if and are probability measures on the same space with densities and , then , which is the likelihood ratio. This is fundamental in: hypothesis testing (Neyman-Pearson lemma), importance sampling (reweighting samples), and KL divergence .
Quick Reference
| Concept | Formula | Key Insight |
|---|---|---|
| σ-Algebra | , closed under complement and countable union | Domain of measurable sets |
| Measure | , countably additive | Assigns "size" to sets |
| Probability Measure | Normalized measure | |
| Lebesgue Integral | Integrates via range slicing | |
| MCT | Swap limit and integral for increasing sequences | |
| Fatou's Lemma | Lower bound on limit of integrals | |
| DCT | Swap limit with domination | |
| Fubini's Theorem | Swap order of integration | |
| Radon-Nikodym | with | Density of one measure w.r.t. another |
| Markov's Inequality | Bound tail probabilities | |
| Chebyshev's Inequality | Bound deviations from mean |
Cross-References
- 096-advanced-tensor-calculus — Integration of tensor fields requires measure-theoretic foundations
- 097-advanced-differential-geometry — Integration on manifolds uses differential forms and measures
- 098-advanced-functional-analysis — spaces are Banach/Hilbert spaces defined via Lebesgue integration
- 100-advanced-topological — Borel σ-algebras connect measure theory to topology
- Probability Theory: All probability axioms are measure-theoretic statements