Text-to-Image Alignment

Module: Generative AI | Difficulty: Advanced

CLIP Contrastive Loss

Cross-Attention in Diffusion

Bootstrapping (Image-Text Pairs from Web)

1. Train CLIP on noisy web data
2. Use CLIP to filter high-quality pairs
3. Train diffusion on filtered data

def clip_loss(image_features, text_features, temperature=0.07):
    image_features = image_features / image_features.norm(dim=1, keepdim=True)
    text_features = text_features / text_features.norm(dim=1, keepdim=True)
    logits = image_features @ text_features.T / temperature
    labels = torch.arange(len(logits), device=logits.device)
    return (nn.functional.cross_entropy(logits, labels) +
            nn.functional.cross_entropy(logits.T, labels)) / 2

Research Insight: The quality gap between text-to-image models comes primarily from data quality, not architecture. DALL-E 3's improvement was largely from better captioning.