Diffusion Models
Diffusion models are the architecture powering modern image generators like Stable Diffusion, DALL-E, and Midjourney. They work by learning to reverse a gradual noising process - like learning to restore a corrupted photo step by step. Think of it as an art restorer who can take a damaged painting and meticulously remove each layer of damage until the original masterpiece is revealed.
Forward Diffusion Process
Reverse Denoising Process
UNet Architecture for Denoising
DDPM Sampling Loop
How Diffusion Models Learn
Imagine you have a pristine photograph. First, you progressively corrupt it by adding static - a little at first, then more until it's pure noise (forward process). The neural network learns to reverse each corruption step. Given a slightly noisy image, it predicts what noise was added, then subtracts that prediction. By chaining thousands of these tiny corrections, pure noise transforms into a coherent image.
The genius is in the simplicity: instead of learning to generate images directly, the model learns the much simpler task of "what noise was added to this image?" This decomposition makes training stable and scalable.
Python Example: Simple Diffusion
import torch
import torch.nn as nn
class SimpleDiffusion:
def __init__(self, T=1000, beta_start=1e-4, beta_end=0.02):
self.T = T
self.betas = torch.linspace(beta_start, beta_end, T)
self.alphas = 1 - self.betas
self.alpha_bars = torch.cumprod(self.alphas, dim=0)
def forward_diffusion(self, x0, t):
noise = torch.randn_like(x0)
alpha_bar_t = self.alpha_bars[t]
xt = torch.sqrt(alpha_bar_t) * x0 + torch.sqrt(1 - alpha_bar_t) * noise
return xt, noise
def denoise_step(self, xt, predicted_noise, t):
alpha_t = self.alphas[t]
alpha_bar_t = self.alpha_bars[t]
beta_t = self.betas[t]
mean = (1 / torch.sqrt(alpha_t)) * (
xt - (beta_t / torch.sqrt(1 - alpha_bar_t)) * predicted_noise
)
if t > 0:
noise = torch.randn_like(xt)
sigma_t = torch.sqrt(beta_t)
return mean + sigma_t * noise
return mean
diffusion = SimpleDiffusion(T=1000)
Hands-on Project: Visualize Noise Schedules
import torch
import matplotlib.pyplot as plt
T = 1000
beta_start, beta_end = 1e-4, 0.02
# Linear schedule
betas_linear = torch.linspace(beta_start, beta_end, T)
alphas_linear = 1 - betas_linear
alpha_bars_linear = torch.cumprod(alphas_linear, dim=0)
# Cosine schedule
steps = torch.arange(T + 1, dtype=torch.float64)
f = torch.cos((steps / T + 0.008) / 1.008 * 3.14159 / 2) ** 2
alpha_bars_cosine = f / f[0]
betas_cosine = 1 - (alpha_bars_cosine[1:] / alpha_bars_cosine[:-1])
fig, axes = plt.subplots(1, 2, figsize=(12, 4))
axes[0].plot(alpha_bars_linear.numpy(), label='Linear')
axes[0].set_title('Signal Remaining Over Time')
axes[0].set_xlabel('Timestep t')
axes[0].set_ylabel('Ξ±Μβ (signal)')
axes[0].legend()
axes[1].plot(alpha_bars_cosine[:-1].numpy(), label='Cosine')
axes[1].set_title('Cosine Schedule')
axes[1].set_xlabel('Timestep t')
axes[1].legend()
plt.tight_layout()
plt.savefig('noise_schedules.png', dpi=150)
plt.show()
Summary
Diffusion models achieve state-of-the-art image generation by learning to reverse a gradual noising process. The forward process corrupts data with Gaussian noise, while the reverse process uses a UNet neural network to predict and remove noise iteratively. Key innovations include learned noise schedules, efficient sampling via DDIM, and conditioning mechanisms for text-guided generation.
Next: We explore GANs, the adversarial framework that revolutionized generative modeling.