GAN Inversion: Encoding Images into Latent Space

Module: Generative AI | Difficulty: Advanced

GAN Inversion Problem

Methods Comparison

| Method | Latent Space | Editing Quality | Reconstruction | |--------|-------------|-----------------|----------------| | Optimization | | High | Excellent | | pSp | | Medium | Good | | e4e | | High | Good | | ReStyle | | High | Excellent |

StyleCLIP

import torch, torch.nn as nn

class pSpEncoder(nn.Module):
    def __init__(self, style_dim=512, n_styles=18):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Conv2d(3, 64, 7, 2, 3), nn.ReLU(),
            nn.Conv2d(64, 128, 4, 2, 1), nn.ReLU(),
            nn.AdaptiveAvgPool2d(8), nn.Flatten(),
            nn.Linear(128*64, 512), nn.ReLU(),
            nn.Linear(512, n_styles * style_dim))
    def forward(self, x):
        return self.encoder(x).view(-1, 18, 512)

Research Insight: space provides better reconstruction than because it allows per-layer style codes, but provides more semantic editing.