Image Generation

Module: Computer Vision | Difficulty: Advanced

Variational Autoencoder (VAE)

Diffusion Models

Forward process adds Gaussian noise:

Reverse process learns to denoise:

Training objective (simplified):

Classifier-Free Guidance

import torch
import torch.nn as nn

class SimpleDiffusion(nn.Module):
    def __init__(self, img_size=64, timesteps=1000):
        super().__init__()
        self.timesteps = timesteps
        self.net = nn.Sequential(
            nn.Conv2d(3, 64, 3, padding=1),
            nn.SiLU(),
            nn.Conv2d(64, 64, 3, padding=1),
            nn.SiLU(),
            nn.Conv2d(64, 3, 3, padding=1),
        )
    
    def forward(self, x_t, t):
        return self.net(x_t)

def diffusion_loss(model, x_0, timesteps=1000):
    noise = torch.randn_like(x_0)
    t = torch.randint(0, timesteps, (x_0.size(0),), device=x_0.device)
    alpha = torch.cumprod(1 - torch.linspace(1e-4, 0.02, timesteps), dim=0)
    x_t = torch.sqrt(alpha[t]).view(-1, 1, 1, 1) * x_0 +           torch.sqrt(1 - alpha[t]).view(-1, 1, 1, 1) * noise
    pred = model(x_t, t)
    return nn.functional.mse_loss(pred, noise)

Key Takeaways

• Diffusion models produce highest-quality generated images
• Classifier-free guidance controls the trade-off between quality and diversity
• Text-to-image models combine vision and language understanding