Adversarial Diffusion Distillation

Module: Generative AI | Difficulty: Advanced

ADD Loss

Three-Stage Training

1. Pre-train diffusion model (teacher)
2. Distill with SDS loss
3. Add adversarial training

SDXL-Turbo

4-step generation with near-GAN quality.

Quality vs Speed Trade-off

def add_loss(generator, discriminator, vae, text_encoder, x0, text):
    t = torch.randint(0, 1000, (x0.size(0),))
    noise = torch.randn_like(x0)
    xt = scheduler.add_noise(vae.encode(x0), noise, t)
    cond = text_encoder(text)
    # SDS component
    with torch.no_grad():
        eps = unet(xt, t, cond)
    sds_grad = eps - noise
    # GAN component
    gen_img = vae.decode(generator(x0.shape, cond))
    fake_score = discriminator(gen_img)
    gan_loss = -fake_score.mean()
    return (sds_grad * xt).mean() + gan_loss

| Method | Steps | FID | CLIP Score | |--------|-------|-----|------------| | SDXL | 50 | 2.3 | 0.32 | | SDXL-Turbo | 4 | 3.1 | 0.31 | | SDXL-Lightning | 4 | 2.8 | 0.30 | | SDXL-Lightning | 2 | 3.5 | 0.29 |

Research Insight: ADD achieves GAN-like speed (1-4 steps) while maintaining diffusion-like quality. The key is that adversarial training provides sharper gradients than SDS alone.