Domain Adaptation

Module: Computer Vision | Difficulty: Advanced

Domain Shift

Source domain differs from target :

Maximum Mean Discrepancy (MMD)

Adversarial Domain Adaptation

CORAL Loss

where is the covariance matrix.

import torch
import torch.nn as nn

class DomainAdversarial(nn.Module):
    def __init__(self, feat_dim, num_classes):
        super().__init__()
        self.classifier = nn.Linear(feat_dim, num_classes)
        self.discriminator = nn.Sequential(
            nn.Linear(feat_dim, 256), nn.ReLU(True),
            nn.Linear(256, 2), nn.Softmax(dim=1),
        )
        self.grl = GradientReversalLayer()
    
    def forward(self, features):
        class_pred = self.classifier(features)
        domain_pred = self.discriminator(self.grl(features))
        return class_pred, domain_pred

class GradientReversalFunction(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x, alpha):
        ctx.alpha = alpha
        return x.view_as(x)
    
    @staticmethod
    def backward(ctx, grad_output):
        return -ctx.alpha * grad_output, None

Key Takeaways

• Domain shift degrades performance on unseen target domains
• Adversarial training aligns feature distributions
• MMD and CORAL provide distribution matching losses