Multi-Task Learning: Shared Representations and Negative Transfer

Module: Machine Learning | Difficulty: Advanced

Hard Parameter Sharing

Soft Parameter Sharing

Negative Transfer

Task Relationship Learning

where is task similarity.

import torch
import torch.nn as nn

class MultiTaskModel(nn.Module):
    def __init__(self, shared_dim, task_dims):
        super().__init__()
        self.shared = nn.Sequential(
            nn.Linear(784, shared_dim), nn.ReLU(),
            nn.Linear(shared_dim, shared_dim))
        self.task_heads = nn.ModuleDict({
            name: nn.Linear(shared_dim, dim)
            for name, dim in task_dims.items()
        })
    def forward(self, x, task_name):
        shared = self.shared(x)
        return self.task_heads[task_name](shared)

Research Insight: Multi-task learning works best when tasks share similar representations. The key insight is that the shared layers learn general features, while task-specific layers learn specialized features. Negative transfer occurs when tasks have conflicting gradients.