Diffusion Inpainting Theory

Module: Generative AI | Difficulty: Advanced

Inpainting Objective

Diffusion Inpainting (RePaint)

1. Forward process: add noise to
2. Reverse process: denoise all pixels
3. Replace known pixels:

Attention Masking

where is the mask.

Proportional Denoising

def inpaint_step(model, x_t, mask, x_known, t, noise_scheduler):
    with torch.no_grad():
        eps = model(x_t, t)
    alpha_t = noise_scheduler.alphas[t]
    alpha_bar_t = noise_scheduler.alphas_cumprod[t]
    # Denoise unknown regions
    x0_pred = (x_t - (1-alpha_bar_t).sqrt() * eps) / alpha_bar_t.sqrt()
    x0_pred = x0_pred.clamp(-1, 1)
    # Combine known and predicted
    x0_combined = mask * x_known + (1-mask) * x0_pred
    return x0_combined

Research Insight: RePaint achieves high-quality inpainting without any training on inpainting data. The key insight is that the diffusion process can be partially applied to known pixels while generating unknown pixels.