Interpolation
Definitions
Formulas
Lagrange Interpolation
Newton Interpolation
Spline Interpolation
Types of Splines
| Type | Continuity | Properties |
|---|---|---|
| Linear | C⁰ | Simple, non-smooth |
| Cubic | C² | Smooth, optimal |
| Natural cubic | C² + S''=0 at endpoints | Smoothest possible |
| Clamped cubic | C² + S' specified at endpoints | Matches derivative |
| Not-a-knot | C³ at interior knots | Smoothest possible C³ |
| Periodic | C² + periodic | For periodic functions |
Examples
Theorems
Python Implementation
import numpy as np
from typing import Callable, List, Tuple
class InterpolationToolkit:
"""Comprehensive interpolation toolkit."""
@staticmethod
def lagrange(x_data: np.ndarray, y_data: np.ndarray,
x: np.ndarray) -> np.ndarray:
"""Lagrange interpolation."""
n = len(x_data)
result = np.zeros_like(x, dtype=float)
for j in range(n):
L_j = np.ones_like(x, dtype=float)
for i in range(n):
if i != j:
L_j *= (x - x_data[i]) / (x_data[j] - x_data[i])
result += y_data[j] * L_j
return result
@staticmethod
def divided_differences(x_data: np.ndarray,
y_data: np.ndarray) -> np.ndarray:
"""Compute divided difference table."""
n = len(x_data)
dd = np.zeros((n, n))
dd[:, 0] = y_data
for j in range(1, n):
for i in range(n - j):
dd[i, j] = (dd[i+1, j-1] - dd[i, j-1]) / (x_data[i+j] - x_data[i])
return dd
@staticmethod
def newton(x_data: np.ndarray, y_data: np.ndarray,
x: np.ndarray) -> np.ndarray:
"""Newton interpolation using Horner's method."""
dd = InterpolationToolkit.divided_differences(x_data, y_data)
n = len(x_data)
result = np.full_like(x, dd[0, n-1], dtype=float)
for k in range(n-2, -1, -1):
result = result * (x - x_data[k]) + dd[0, k]
return result
@staticmethod
def natural_cubic_spline(x_data: np.ndarray, y_data: np.ndarray,
x_eval: np.ndarray) -> np.ndarray:
"""Natural cubic spline interpolation."""
n = len(x_data) - 1
h = np.diff(x_data)
# Set up tridiagonal system for moments
A = np.zeros((n+1, n+1))
b = np.zeros(n+1)
A[0, 0] = 1
A[n, n] = 1
for i in range(1, n):
A[i, i-1] = h[i-1]
A[i, i] = 2 * (h[i-1] + h[i])
A[i, i+1] = h[i]
b[i] = 6 * ((y_data[i+1] - y_data[i]) / h[i] -
(y_data[i] - y_data[i-1]) / h[i-1])
m = np.linalg.solve(A, b)
# Evaluate
results = np.zeros_like(x_eval, dtype=float)
for j, x in enumerate(x_eval):
i = np.searchsorted(x_data, x, side='right') - 1
i = max(0, min(i, n-1))
dx = x - x_data[i]
hi = h[i]
A_c = (m[i+1] - m[i]) / (6 * hi)
B_c = m[i] / 2
C_c = (y_data[i+1] - y_data[i]) / hi - (m[i+1] + 2*m[i]) * hi / 6
D_c = y_data[i]
results[j] = A_c * dx**3 + B_c * dx**2 + C_c * dx + D_c
return results
@staticmethod
def chebyshev_nodes(a: float, b: float, n: int) -> np.ndarray:
"""Generate Chebyshev nodes on [a, b]."""
k = np.arange(n)
nodes = np.cos((2*k + 1) * np.pi / (2*n))
return 0.5 * (b - a) * nodes + 0.5 * (b + a)
@staticmethod
def linear_spline(x_data: np.ndarray, y_data: np.ndarray,
x_eval: np.ndarray) -> np.ndarray:
"""Linear spline (piecewise linear interpolation)."""
return np.interp(x_eval, x_data, y_data)
def demonstrate_interpolation():
"""Comprehensive interpolation demonstration."""
print("=" * 60)
print("INTERPOLATION METHOD COMPARISON")
print("=" * 60)
toolkit = InterpolationToolkit()
# Test function
f = lambda x: np.sin(x) + 0.5 * np.cos(2 * x)
x_data = np.linspace(0, 2*np.pi, 8)
y_data = f(x_data)
x_fine = np.linspace(0, 2*np.pi, 200)
y_true = f(x_fine)
print("\n--- Method Comparison (8 data points) ---")
print(f"{'Method':<25} {'Max Error':<14}")
print("-" * 39)
y_lagrange = toolkit.lagrange(x_data, y_data, x_fine)
print(f"{'Lagrange':<25} {np.max(np.abs(y_lagrange - y_true)):<14.6e}")
y_newton = toolkit.newton(x_data, y_data, x_fine)
print(f"{'Newton':<25} {np.max(np.abs(y_newton - y_true)):<14.6e}")
y_spline = toolkit.natural_cubic_spline(x_data, y_data, x_fine)
print(f"{'Cubic Spline':<25} {np.max(np.abs(y_spline - y_true)):<14.6e}")
y_linear = toolkit.linear_spline(x_data, y_data, x_fine)
print(f"{'Linear Spline':<25} {np.max(np.abs(y_linear - y_true)):<14.6e}")
# Chebyshev nodes comparison
print("\n--- Chebyshev vs Equally-Spaced (Runge function) ---")
runge = lambda x: 1 / (1 + 25*x**2)
for n in [5, 10, 15]:
# Equally spaced
x_eq = np.linspace(-1, 1, n)
y_eq = runge(x_eq)
y_eq_interp = toolkit.lagrange(x_eq, y_eq, x_fine)
err_eq = np.max(np.abs(y_eq_interp - runge(x_fine)))
# Chebyshev
x_cheb = toolkit.chebyshev_nodes(-1, 1, n)
y_cheb = runge(x_cheb)
y_cheb_interp = toolkit.lagrange(x_cheb, y_cheb, x_fine)
err_cheb = np.max(np.abs(y_cheb_interp - runge(x_fine)))
print(f" n={n:>2}: Equally-spaced error = {err_eq:.6f}, "
f"Chebyshev error = {err_cheb:.6f}")
demonstrate_interpolation()
Applications in AI/ML
Specific Applications
| Application | Method | Description |
|---|---|---|
| Gaussian processes | Kernel interpolation | Predict at new points using kernel-weighted average |
| RBF networks | Radial basis interpolation | Neural network with RBF activation functions |
| Data augmentation | Spline interpolation | Generate synthetic training examples |
| Feature engineering | Polynomial interpolation | Create interaction features |
| Time series | Spline smoothing | Reconstruct signals from irregular samples |
| Transfer learning | Function interpolation | Interpolate between model weights |
| Curriculum learning | Data interpolation | Interpolate between easy and hard examples |
Neural Tangent Kernel
In the infinite-width limit, neural networks perform kernel interpolation with the neural tangent kernel. The interpolating function is the minimum-norm solution in the RKHS (reproducing kernel Hilbert space) that passes through all training points.
Weight Space Interpolation
SLERP (Spherical Linear Interpolation) interpolates between model weight vectors on the unit hypersphere, used in fine-tuning and model merging.
Common Mistakes
| Mistake | Problem | Solution |
|---|---|---|
| Using high-degree polynomial | Runge's phenomenon causes oscillation | Use splines or Chebyshev nodes |
| Equally-spaced nodes for interpolation | Error grows exponentially near endpoints | Use Chebyshev or clustering nodes |
| Extrapolating far beyond data | Unreliable, often wildly wrong | Use domain knowledge; prefer interpolation |
| Not checking data monotonicity | Spline behavior may be unexpected | Verify x_data is sorted and unique |
| Using polynomial for periodic data | Polynomial can't capture periodicity | Use Fourier or periodic splines |
| Ignoring outlier data points | Polynomial/spline follows outliers | Use robust interpolation or pre-filter |
| Too few data points | Underfitting; missing features | Increase sampling density |
| Using spline for extrapolation | Splines extrapolate as cubic (often wrong) | Restrict to interpolation domain |
Interview Questions
Q1: What is Runge's phenomenon and how do you avoid it? A: Runge's phenomenon occurs when high-degree polynomial interpolation on equally-spaced nodes oscillates wildly near interval endpoints. For f(x) = 1/(1+25x²), the error grows as n increases. Solutions: (1) use cubic splines instead of high-degree polynomials, (2) use Chebyshev nodes (non-uniform spacing), (3) reduce the polynomial degree and accept piecewise approximation.
Q2: Compare Lagrange and Newton interpolation. A: Both produce the same polynomial (uniqueness theorem). Lagrange is theoretically cleaner but O(n²) to add a new point. Newton uses divided differences — O(n²) to build but O(n) to evaluate and incrementally updatable. Newton is preferred for computation; Lagrange for theory.
Q3: Why are cubic splines preferred over high-degree polynomials? A: Cubic splines use piecewise low-degree polynomials, avoiding Runge's phenomenon. They ensure C² continuity (smooth visually), have optimal approximation properties, and scale linearly with the number of data points. High-degree polynomials oscillate and are expensive to evaluate.
Q4: How does a natural cubic spline differ from a clamped cubic spline? A: Natural cubic spline: S''(x₀) = S''(xₙ) = 0 (second derivative zero at endpoints). Clamped cubic spline: S'(x₀) and S'(xₙ) are specified (usually matching the known derivative). Natural splines are smoother; clamped splines better approximate the true function when derivatives are known.
Q5: When is extrapolation appropriate? A: Extrapolation is appropriate only when: (1) you have strong physical/theoretical reasons to believe the trend continues, (2) the extrapolation distance is small relative to the data range, (3) the function is known to be monotone or have known asymptotic behavior. Never extrapolate black-box functions.
Q6: Explain the connection between interpolation and approximation theory. A: Interpolation requires the approximant to pass exactly through data points, while approximation (e.g., least squares) minimizes overall error. Interpolation is a special case of approximation. The key trade-off: interpolation has zero error at nodes but may oscillate between them; approximation distributes error evenly but doesn't match exactly.
Practice Problems
Quick Reference
Cross-References
- 086-Floating-Point — Polynomial evaluation in interpolation requires careful attention to floating point error; Horner's method minimizes operations and error
- 087-Root-Finding — Divided differences in Newton interpolation are computed via recursive formulas similar to secant method differences
- 088-Integration — Quadrature rules are derived by integrating interpolating polynomials; Gaussian quadrature uses optimal node placement from approximation theory
- 090-Error Analysis — Interpolation error analysis depends on derivatives of the function and the distribution of nodes
- Approximation Theory: Interpolation is a special case of function approximation; the Weierstrass theorem guarantees polynomial approximation exists
- Signal Processing: Shannon sampling theorem relates interpolation to signal reconstruction; Nyquist rate determines minimum sampling density