Prerequisites

Before diving into GANs, you should be comfortable with:

• Neural Network Fundamentals: Forward/backward pass, activation functions, backpropagation (see Tutorial 21)
• CNNs: Transposed convolutions (deconvolution), convolutional architectures (see Tutorial 22)
• Probability: Minimax games, Nash equilibrium, probability distributions
• Loss Functions: Cross-entropy, binary cross-entropy
• PyTorch: Training loops, gradient computation, batch normalization

Learning Objectives

By the end of this tutorial, you will be able to:

1. Explain the minimax game between generator and discriminator
2. Derive the GAN loss function and its optimal solution
3. Implement DCGAN with transposed convolutions for image generation
4. Identify and mitigate mode collapse in GAN training
5. Understand Wasserstein GAN and its advantages over vanilla GAN
6. Compare GAN variants (cGAN, Pix2Pix, CycleGAN, StyleGAN) for different tasks
7. Implement a basic GAN training loop in PyTorch

GANs — Generative Adversarial Networks

GANs (Goodfellow et al., 2014) learn to generate data by framing generation as a two-player minimax game between a generator and discriminator .

Loss Functions

Training Process

GAN training alternates between optimizing D and G:

Mode Collapse

Mode collapse occurs when G learns to produce only a few outputs that reliably fool D, ignoring the full data distribution.

DCGAN Architecture

WGAN (Wasserstein GAN)

Real-World Applications

Common Mistakes & How to Avoid Them

Key Formulas Reference

Interview Questions

Practice Exercise

import torch
import torch.nn as nn

# Task 1: Implement DCGAN Generator (noise -> 28x28 image)
class Generator(nn.Module):
    def __init__(self, latent_dim=100):
        super().__init__()
        # Use ConvTranspose2d layers
        # 100 -> 256x7x7 -> 128x14x14 -> 1x28x28
        pass

    def forward(self, z):
        pass

# Task 2: Implement DCGAN Discriminator (image -> real/fake)
class Discriminator(nn.Module):
    def __init__(self):
        super().__init__()
        # Use Conv2d layers with stride 2 for downsampling
        pass

    def forward(self, x):
        pass

# Task 3: Training loop with:
# - Non-saturating loss for G
# - BCE loss for D
# - Learning rates: G=2e-4, D=1e-4 (Adam, beta1=0.5)
# - Label smoothing: real=0.9, fake=0.1

# Task 4: Generate and save a grid of 64 samples every 10 epochs

Key Takeaways

What to Learn Next

Advanced Topics

Progressive Growing (ProGAN)

ProGAN (Karras et al., 2017) trains GANs by progressively increasing resolution:

1. Start with 4x4 images (1 residual block)
2. Train until stable, then add layers to double resolution
3. Fade in new layers smoothly using a parameter that ramps from 0 to 1

Why it works: Low-resolution images are easier to generate, so the model learns coarse structure first, then refines details. This enables training 1024x1024 generators that would otherwise be impossible.

StyleGAN Architecture

StyleGAN separates "style" from "content":

1. Mapping network: Maps latent vector to intermediate latent space
2. Style modulation: Each layer receives a learned affine transformation of
3. Noise injection: Per-pixel noise adds stochastic variation (hair texture, freckles)

Key innovation: Styles are injected at different layers — coarse styles (pose, face shape) at early layers, fine styles (skin texture, lighting) at later layers.

GANs vs Diffusion Models

Conditional Generation

Conditional GANs (cGAN) add auxiliary information to control generation:

Applications: Text-to-image (DALL-E, Midjourney), class-conditional generation, image-to-image translation (Pix2Pix).

Evaluation Metrics

• FID (Fréchet Inception Distance): Compares statistics of real vs generated images in Inception-v3 feature space. Lower = better. The most widely used metric.
• IS (Inception Score): Measures quality (sharp images) and diversity (various classes). Higher = better. Less reliable than FID.
• KID (Kernel Inception Distance): Bias-corrected version of FID. Better for small sample sizes.
• Precision/Recall: Precision measures quality (are generated images realistic?), Recall measures diversity (do they cover all modes?).

Comparison Table

Further Reading

• Goodfellow, I. et al. (2014). "Generative Adversarial Networks." — The original GAN paper that started the field.
• Radford, A. et al. (2015). "Unsupervised Representation Learning with Deep Convolutional GANs." — DCGAN architecture.
• Arjovsky, M. et al. (2017). "Wasserstein GAN." — Stable training with Wasserstein distance.
• Karras, T. et al. (2018). "Progressive Growing of GANs." — Training high-resolution generators.
• Karras, T. et al. (2019). "A Style-Based Generator Architecture for GANs." — StyleGAN with style injection.
• Salimans, T. et al. (2016). "Improved Techniques for Training GANs." — Practical tips for stable GAN training.

Quick Reference Cheat Sheet