Tensor Products
For matrices and , the Kronecker product is:
The result is a block matrix where each block is a submatrix scaled by .
Matrix Exponential
For any square matrix :
This series converges absolutely for all matrices .
Matrix Logarithm
The matrix logarithm is the inverse of the matrix exponential: if , then . It exists if and only if is invertible and has no eigenvalues on the negative real axis (principal logarithm).
Matrix Powers
Generalized Eigenvalue Problem
Given two square matrices , find scalars and non-zero vectors satisfying . This reduces to the standard eigenvalue problem when .
Matrix Functions
For a function and a diagonalizable matrix :
This extends scalar functions to matrices: , , , , , , and fractional powers .
Quadratic Forms
For a symmetric matrix , the quadratic form maps to a scalar:
Rayleigh Quotient
Polar Decomposition
Any square matrix can be decomposed as:
When is invertible, is unique and is positive definite. The left polar decomposition is where .
Jordan Normal Form
When a matrix is defective (lacks enough eigenvectors for diagonalization), the Jordan normal form provides the closest possible simplification:
Each Jordan block has the form:
Python Implementation
Applications in AI/ML
Common Mistakes
| Mistake | Correction |
|---|---|
| Assuming always | This only holds when ; for non-commuting matrices, use the Baker-Campbell-Hausdorff formula |
| Using Jordan form for numerical computation | Jordan form is numerically unstable; use Schur decomposition for floating-point arithmetic |
| Treating tensor product as element-wise multiplication | produces a block matrix of size , not which requires compatible dimensions |
| Assuming always | The matrix logarithm is multi-valued; for integer in the complex case |
| Ignoring positive definiteness in quadratic forms | Always verify eigenvalues of before concluding a critical point is a minimum; positive semi-definite is not sufficient |
| Confusing algebraic and geometric multiplicity | Geometric multiplicity algebraic multiplicity; they differ for defective matrices |
| Forgetting that polar decomposition requires to be PSD | must be the positive semi-definite square root, not just any square root |
Interview Questions
Practice Problems
Quick Reference
| Concept | Formula | Key Property |
|---|---|---|
| Kronecker Product | Size , eigenvalues | |
| Matrix Exponential | ||
| Matrix Logarithm | Inverse of | |
| Matrix Power | via diagonalization | |
| Generalized Eigen | ||
| Matrix Function | Applies to each eigenvalue | |
| Quadratic Form | Positive definite iff all | |
| Rayleigh Quotient | ||
| Polar Decomposition | Rotation + Scaling | |
| Jordan Normal Form | Handles defective matrices |
Cross-References
- Eigenvalues and Eigenvectors: Foundation for all matrix functions β enables
- Singular Value Decomposition: is the polar decomposition of when written as
- Positive Definite Matrices: Quadratic forms and Rayleigh quotients rely on positive definiteness
- Matrix Calculus: Derivatives of matrix exponentials and quadratic forms appear in optimization
- Optimization: The Hessian determines whether critical points are minima/maxima via quadratic form classification
- Probability and Statistics: Covariance matrices are positive semi-definite; Mahalanobis distance uses quadratic forms
- Numerical Methods: Pade approximation computes ; Schur decomposition avoids Jordan form instability
- Discrete Mathematics: Graph Laplacians and their eigenvalues power spectral graph theory and GNNs