Dimensionality Reduction: PCA, t-SNE, and UMAP

Module: Machine Learning | Difficulty: Advanced

PCA

Principal components: columns of .

Explained Variance

t-SNE

UMAP

import numpy as np
from sklearn.decomposition import PCA

class PCAManual:
    def __init__(self, n_components=2):
        self.n_components = n_components
    def fit_transform(self, X):
        X_centered = X - X.mean(axis=0)
        cov = np.cov(X_centered.T)
        eigenvalues, eigenvectors = np.linalg.eigh(cov)
        idx = np.argsort(eigenvalues)[::-1]
        self.components = eigenvectors[:, idx[:self.n_components]]
        self.explained_variance = eigenvalues[idx[:self.n_components]]
        return X_centered @ self.components

| Method | Preserves | Speed | Interpretability | |--------|-----------|-------|------------------| | PCA | Global structure | Fast | High | | t-SNE | Local structure | Slow | Low | | UMAP | Both | Medium | Low | | Isomap | Geodesic | Medium | Medium |

Research Insight: PCA is optimal for linear manifolds but fails for nonlinear ones. t-SNE and UMAP are better for visualization but don't scale well. The choice depends on whether you need global or local structure preservation.