Generative Models
VAEs — Learning Probabilistic Latent Representations
VAEs combine autoencoders with variational inference to learn smooth, continuous latent spaces. By encoding to distributions instead of points, they enable principled generation, meaningful interpolation, and disentangled representations — all with a stable training objective.
- Key point 1 — ELBO loss balances reconstruction quality with latent space regularization
- Key point 2 — Reparameterization trick enables backpropagation through stochastic sampling
- Key point 3 — Beta-VAE and VQ-VAE extend the framework for disentanglement and discrete latents
"In the latent space, every point tells a story."
Variational Autoencoders — Deep Dive
VAEs are generative models that learn a smooth latent space by combining autoencoders with variational inference. Unlike GANs, they provide a principled probabilistic framework for generation.
From Autoencoder to VAE
How this diagram works: This diagram shows the VAE architecture, which differs from a standard autoencoder by encoding inputs to a probability distribution rather than a fixed point. The encoder (green) maps input to parameters of a Gaussian distribution — a mean and variance . Instead of sampling directly (which is non-differentiable), the reparameterization trick samples where , making the process differentiable for backpropagation. The decoder (red) then reconstructs the input from this sampled latent vector. The loss combines reconstruction quality with a KL divergence term that regularizes the latent space toward a standard Gaussian prior, ensuring it remains smooth and continuous — enabling meaningful interpolation and generation by sampling from the prior.