Catastrophic Forgetting and Continual Learning

Module: Machine Learning | Difficulty: Advanced

Catastrophic Forgetting

Training on task B destroys performance on task A:

EWC (Elastic Weight Consolidation)

where is the Fisher information.

Experience Replay

Progressive Neural Networks

Add new columns for each task, freeze old columns.

import torch
import torch.nn as nn

class EWC:
    def __init__(self, model, lambda_=1000):
        self.model = model; self.lambda_ = lambda_
        self.fisher = {}; self.theta_old = {}
    def compute_fisher(self, data_loader):
        for name, param in self.model.named_parameters():
            self.fisher[name] = torch.zeros_like(param)
            self.theta_old[name] = param.data.clone()
        for x, y in data_loader:
            self.model.zero_grad()
            loss = nn.functional.cross_entropy(self.model(x), y)
            loss.backward()
            for name, param in self.model.named_parameters():
                self.fisher[name] += param.grad.data ** 2
        for name in self.fisher:
            self.fisher[name] /= len(data_loader)
    def penalty(self):
        loss = 0
        for name, param in self.model.named_parameters():
            loss += (self.fisher[name] * (param - self.theta_old[name])**2).sum()
        return self.lambda_ * loss

Research Insight: EWC's key insight is that important parameters for task A (high Fisher information) should not change when learning task B. This approximates the optimal solution under the assumption that the posterior over parameters is Gaussian.