Meta-Learning Theory: Generalization Bounds and Task Distributions

Module: Machine Learning | Difficulty: Advanced

PAC-Bayes for Meta-Learning

Task Similarity

Algorithm Design Principles

import numpy as np

def task_similarity(tasks, model):
    grads = []
    for task in tasks:
        grad = compute_grad(model, task)
        grads.append(grad)
    grad_array = np.array(grads)
    sim_matrix = grad_array @ grad_array.T
    sim_matrix = sim_matrix / np.sqrt(np.outer(np.diag(sim_matrix), np.diag(sim_matrix)))
    return sim_matrix

Research Insight: MAML's generalization bound depends on the task distribution similarity. When tasks are similar, MAML achieves strong generalization; when tasks are diverse, the bound becomes vacuous. Task clustering can improve performance.