Integration Fundamentals
What is an Integral
Fundamental Theorem of Calculus
Properties of Definite Integrals
| Property | Formula | Description |
|---|
| Reversing bounds | | Swapping limits negates the integral |
| Additivity | | Split at any intermediate point |
| Linearity | | Constants factor out, integrals add |
| Zero-width | | Integrating over a point gives zero |
| Positivity | If on , then | Non-negative functions have non-negative integrals |
| Comparison | If on , then | Inequality preserved under integration |
| Triangle Inequality | | The integral of the absolute value bounds the absolute value of the integral |
Basic Integration Rules
The following rules allow us to integrate complex functions by breaking them into simpler parts.
| Rule | Formula | Example |
|---|
| Constant | | |
| Power Rule | () | |
| Reciprocal | | |
| Exponential | | |
| General Exponential | | |
| Sine | | |
| Cosine | | |
| Secant | | |
| Cosecant | | |
| Secant-Tangent | | |
| Cosecant-Cotangent | | |
| Inverse Sine | | |
| Inverse Tangent | | |
| Hyperbolic Sine | | |
| Hyperbolic Cosine | | |
Integration by Substitution
Integration by Parts
Common Integrals
| Integral | Result | Notes |
|---|
| | |
| | The case |
| | Its own antiderivative |
| | , |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | Rewrite as |
| | |
| | |
| | |
| | Partial fractions |
| | |
| | |
| | |
| | |
| | |
Improper Integrals
Numerical Integration
When an antiderivative cannot be found in closed form, or when the integrand is defined only by data points, we approximate the integral numerically.
Trapezoidal Rule
Simpson's Rule
Gaussian Quadrature
| Method | Order of Accuracy | Best For | Nodes Required |
|---|
| Trapezoidal | | Rough data, quick estimates | Uniform grid |
| Simpson's | | Smooth functions, moderate precision | Uniform grid (even ) |
| Gaussian | | High-precision integration of smooth functions | Optimally placed nodes |
| Monte Carlo | | High-dimensional integrals | Random samples |
Python Implementation
Basic Integration with scipy.integrate
import numpy as np
from scipy import integrate
# Define the function to integrate
def f(x):
return x**2 * np.exp(-x)
# Compute the integral from 0 to infinity (improper integral)
result, error = integrate.quad(f, 0, np.inf)
print(f"Integral of x^2 * exp(-x) from 0 to inf:")
print(f" Result: {result:.6f}")
print(f" Error estimate: {error:.2e}")
print(f" Exact (2! = 2): 2.000000")
# Definite integral from 0 to 1
result, error = integrate.quad(lambda x: np.sin(x), 0, np.pi)
print(f"\nIntegral of sin(x) from 0 to pi: {result:.6f} (exact: 2)")
Numerical Methods Comparison
import numpy as np
from scipy import integrate
# Define test function: f(x) = x^2
f = lambda x: x**2
a, b = 0, 1
exact = 1/3
# Trapezoidal rule
for n in [10, 100, 1000]:
x = np.linspace(a, b, n + 1)
trap_result = np.trapz(f(x), x)
print(f"Trapezoidal (n={n:4d}): {trap_result:.8f} error: {abs(trap_result - exact):.2e}")
print()
# Simpson's rule
for n in [10, 100, 1000]:
x = np.linspace(a, b, n + 1)
simp_result = integrate.simpson(f(x), x=x)
print(f"Simpson's (n={n:4d}): {simp_result:.8f} error: {abs(simp_result - exact):.2e}")
print()
# Gaussian quadrature (scipy)
for n in [5, 10, 20]:
result, error = integrate.fixed_quad(lambda x: x**2, a, b, n=n)
print(f"Gauss quad (n={n:4d}): {result:.8f} error: {abs(result - exact):.2e}")
# scipy.integrate.quad (adaptive)
result, error = integrate.quad(f, a, b)
print(f"\nAdaptive quad: {result:.8f} error: {error:.2e}")
Symbolic Integration with SymPy
import sympy as sp
x = sp.Symbol('x')
# Symbolic indefinite integral
f = x**2 * sp.exp(x)
F = sp.integrate(f, x)
print(f"Indefinite integral of x^2 * e^x: {F}")
# Definite integral
result = sp.integrate(x**2, (x, 0, 1))
print(f"Definite integral of x^2 from 0 to 1: {result}")
# Improper integral
result = sp.integrate(sp.exp(-x**2), (x, -sp.oo, sp.oo))
print(f"Integral of e^(-x^2) from -inf to inf: {result}")
print(f" = sqrt(pi) = {sp.sqrt(sp.pi)}")
# Verify Fundamental Theorem
F = sp.integrate(sp.sin(x), x)
print(f"\nAntiderivative of sin(x): {F}")
print(f"FTC: F(pi) - F(0) = {F.subs(x, sp.pi) - F.subs(x, 0)}")
High-Dimensional Integration (Monte Carlo)
import numpy as np
def monte_carlo_integrate(f, bounds, n_samples=100000):
"""Monte Carlo integration for arbitrary dimensions."""
dim = len(bounds)
samples = np.random.uniform(
low=[b[0] for b in bounds],
high=[b[1] for b in bounds],
size=(n_samples, dim)
)
values = np.array([f(s) for s in samples])
volume = np.prod([b[1] - b[0] for b in bounds])
mean_val = np.mean(values)
std_err = np.std(values) / np.sqrt(n_samples)
return mean_val * volume, std_err
# Example: integral of x^2 + y^2 over [0,1] x [0,1]
f = lambda p: p[0]**2 + p[1]**2
result, error = monte_carlo_integrate(f, [(0, 1), (0, 1)])
exact = 2/3 # integral of x^2 + y^2 over [0,1]^2
print(f"Monte Carlo estimate: {result:.6f} +/- {error:.6f}")
print(f"Exact value: {exact:.6f}")
Applications in AI/ML
Probability Density Functions
Expected Values and Moments
Bayesian Inference
Common Probability Integrals
| Distribution | PDF | Key Integral |
|---|
| Normal | | |
| Standard Normal | | |
| Exponential | , | |
| Beta | | |
| Gamma | | |
Common Mistakes
| Mistake | Incorrect | Correct | Explanation |
|---|
| Forgetting the constant of integration | | | Always include for indefinite integrals |
| Wrong sign on trig integral | | | The integral of sine is negative cosine |
| Power rule on | | | Power rule fails at ; use |
| Not changing bounds on substitution | Evaluate with old bounds after -sub | Update bounds when substituting | If , new bounds are and |
| Dropping absolute value in | | | is defined for too |
| Confusing integration with differentiation rules | Treating integrals like derivatives | Integration follows different rules | E.g., |
| Forgetting boundary term in integration by parts | | | The term is essential |
| Divergent improper integrals | Assuming always converges | Converges only for | Check convergence before evaluating |
| Wrong in substitution | Forgetting to include the derivative | not just | The differential must include the derivative |
| Splitting integrals incorrectly | (by symmetry) | diverges | The integrand has a singularity at |
Interview Questions
Q1: State the Fundamental Theorem of Calculus and explain its significance.
Q2: When would you use integration by parts vs. substitution?
Q3: Why does ? Why is this important in ML?
Q4: Explain the difference between convergence and divergence for improper integrals. Give an example of each.
Q5: How is numerical integration used when analytical solutions are unavailable?
Q6: What role does integration play in training probabilistic models?
Q7: Prove that using the definition of the integral.
Practice Problems
Quick Reference
Cross-References
- Limits and Continuity: The foundation for understanding integrals as limits of Riemann sums -> Limits and Continuity
- Derivatives and Differentiation: Integration is the reverse operation of differentiation -> Derivatives and Differentiation
- Chain Rule: Integration by substitution is the reverse of the chain rule -> Chain Rule and Implicit Differentiation
- Multivariable Calculus: Double and triple integrals extend integration to multiple dimensions -> Multivariable Calculus
- Taylor Series: Polynomial approximations used in deriving integration rules -> Taylor Series
- Probability Foundations: Integration is the backbone of continuous probability -> Probability Foundations
- Probability Distributions: Common distributions and their integral properties -> Probability Distributions
- Expectation and Variance: Expected values are integrals of functions against PDFs -> Expectation and Variance
- Numerical Integration: In-depth coverage of numerical methods -> Numerical Methods
- Differential Equations: Many differential equations are solved by integration -> Differential Equations
- Optimization: Integration in the context of optimization and Lagrange multipliers -> Optimization Fundamentals
- Statistics (MLE): MLE involves integrals over likelihood functions -> Maximum Likelihood Estimation
- Bayesian Statistics: Posterior computation requires integration -> Bayesian Statistics