Limits and Continuity

Limits and Continuity

What is a Limit?

Limit Laws

One-Sided Limits

Infinite Limits and Limits at Infinity

Common Limits

Squeeze Theorem

then .

L'Hôpital's Rule

Continuity

Intermediate Value Theorem

Limits and Derivatives

Python Implementation: Numerical Verification of Limits

import numpy as np

def verify_limit(func, a, expected, approach='both', tol=1e-8):
    """Numerically verify that lim_{x -> a} f(x) = expected."""
    results = {}

    if approach in ('both', 'left'):
        x_left = a - np.array([1e-1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6])
        x_left = x_left[x_left != a]  # exclude a itself
        vals_left = [func(x) for x in x_left]
        results['left'] = vals_left

    if approach in ('both', 'right'):
        x_right = a + np.array([1e-1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6])
        vals_right = [func(x) for x in x_right]
        results['right'] = vals_right

    if approach == 'both':
        final = (np.mean(vals_left[-2:]) + np.mean(vals_right[-2:])) / 2
    elif approach == 'left':
        final = np.mean(vals_left[-2:])
    else:
        final = np.mean(vals_right[-2:])

    return final, abs(final - expected) < tol


# Example 1: sin(x)/x -> 1
val, ok = verify_limit(lambda x: np.sin(x) / x if x != 0 else 1.0, a=0, expected=1.0)
print(f"lim x->0 sin(x)/x = {val:.10f}  (expected 1.0)  {'PASS' if ok else 'FAIL'}")

# Example 2: (e^x - 1)/x -> 1
val, ok = verify_limit(lambda x: (np.exp(x) - 1) / x if x != 0 else 1.0, a=0, expected=1.0)
print(f"lim x->0 (e^x-1)/x = {val:.10f}  (expected 1.0)  {'PASS' if ok else 'FAIL'}")

# Example 3: (1 + 1/n)^n -> e
def compound(n):
    return (1 + 1/n) ** n if n != 0 else np.e

val, ok = verify_limit(compound, a=10000, expected=np.e, approach='right')
print(f"lim n->inf (1+1/n)^n = {val:.10f}  (expected {np.e:.10f})  {'PASS' if ok else 'FAIL'}")

# Example 4: Numerical derivative (limit definition)
def f(x):
    return x**2

h_values = np.array([1e-1, 1e-2, 1e-3, 1e-4, 1e-5])
numerical_derivs = [(f(1 + h) - f(1)) / h for h in h_values]
print(f"\nNumerical derivative of x^2 at x=1:")
for h, d in zip(h_values, numerical_derivs):
    print(f"  h={h:.0e}: {d:.10f}")
print(f"  Expected: 2.0")

# Example 5: Squeeze theorem verification — x^2 sin(1/x)
def squeeze_func(x):
    return x**2 * np.sin(1/x) if x != 0 else 0.0

val, ok = verify_limit(squeeze_func, a=0, expected=0.0)
print(f"\nlim x->0 x^2*sin(1/x) = {val:.10f}  (expected 0.0)  {'PASS' if ok else 'FAIL'}")

Applications in AI/ML

Gradient Descent Convergence

Asymptotic Analysis

Limits allow us to compare algorithm complexity:

• vs : we evaluate , confirming is asymptotically larger.
• Training time for large models: determines cost scaling.

Convergence of Loss Functions

Probability and Statistics

• Law of Large Numbers: (sample mean converges to population mean)
• Central Limit Theorem: Distributional limits underpin confidence intervals
• Bayesian posterior: (posterior concentrates at true parameter)

Neural Network Expressiveness

Common Mistakes

Interview Questions

Q1: What is the epsilon-delta definition of a limit, and why is it needed?

A: The epsilon-delta definition provides a rigorous foundation for limits. It eliminates ambiguity by formalizing "approaches" with precise tolerances. Intuitive notions of limits fail for pathological functions (like near 0). The epsilon-delta definition is needed to prove limit laws, establish the correctness of L'Hôpital's Rule, and build calculus on solid logical foundations.

Q2: When does exist even though direct substitution fails?

A: When both and (or both ), we have an indeterminate form. The limit may still exist. Techniques include:

• Factor and cancel common factors
• Apply L'Hôpital's Rule (differentiate top and bottom)
• Use series expansion (Taylor series)
• Use the Squeeze Theorem

Q3: Explain the relationship between one-sided limits and continuity.

A: A function is continuous at if and only if:

1. The left-hand limit exists
2. The right-hand limit exists
3. Both are equal to

If the one-sided limits exist but differ, there is a jump discontinuity. If one or both don't exist, continuity fails.

Q4: Why can't we just use L'Hôpital's Rule for every limit problem?

A: L'Hôpital's Rule only applies to indeterminate forms ( or ). Applying it to non-indeterminate forms gives incorrect results. For example, , but applying L'Hôpital gives , which is wrong. Additionally, L'Hôpital requires the derivatives to exist and the limit of the ratio of derivatives to exist.

Q5: How does the Squeeze Theorem help in machine learning?

A: The Squeeze Theorem is used to:

• Prove convergence of algorithms when direct evaluation is difficult
• Establish bounds on error terms in numerical methods
• Show that noise terms vanish: if you can bound the noise between two functions that both go to 0, the noise itself vanishes
• Prove that regularized loss functions converge to their unregularized counterparts as the regularization parameter goes to 0

Q6: What happens when you take the limit of a sequence of functions? Is it always continuous?

A: No. The limit of continuous functions can be discontinuous. Consider on : each is continuous, but equals 0 for and 1 at , which is discontinuous. Uniform convergence (a stronger condition than pointwise convergence) preserves continuity.

Practice Problems

Quick Reference

Cross-References

• 025 — Derivatives and Differentiation: The derivative is defined as a limit of the difference quotient.
• 026 — Chain Rule: Chain rule relies on limits of composite functions.
• 027 — Partial Derivatives: Multivariable limits and partial derivatives.
• 028 — Integrals: Integrals are defined as limits of Riemann sums.
• 029 — Taylor Series: Taylor series convergence depends on limit behavior.
• 030 — Optimization: Optimization requires limits to define derivatives and find critical points.
• 062 — Gradient Descent: Convergence proofs use limit theory.
• 042 — Central Limit Theorem: Distributional limits in probability.