Domain Adaptation: Bridging Source and Target Domains

Module: Machine Learning | Difficulty: Advanced

Domain Adaptation Setting

Source: , Target:

H-Divergence

MMD (Maximum Mean Discrepancy)

Domain-Adversarial Neural Network (DANN)

Theory

where is the joint ideal error.

import torch
import torch.nn as nn

class DomainAdversarial(nn.Module):
    def __init__(self, feature_extractor, classifier, domain_classifier):
        super().__init__()
        self.feature = feature_extractor
        self.classifier = classifier
        self.domain = domain_classifier
        self.grl = GradientReversalLayer()
    def forward(self, x):
        features = self.feature(x)
        class_pred = self.classifier(features)
        domain_pred = self.domain(self.grl(features))
        return class_pred, domain_pred

class GradientReversalFunction(torch.autograd.Function):
    @staticmethod
    def forward(ctx, x, alpha):
        ctx.alpha = alpha
        return x.view_as(x)
    @staticmethod
    def backward(ctx, grad_output):
        return -ctx.alpha * grad_output, None

Research Insight: The key insight in domain adaptation is that features that are invariant across domains are useful for transfer. Adversarial training enforces this invariance, but can hurt performance if the domains have different label distributions.