Meta-Learning — Learning to Learn

Meta-learning trains models to learn new tasks quickly from few examples.

The Problem

Standard ML: Learn ONE task from MANY examples
Meta-Learning: Learn MANY tasks from FEW examples each

Example:
Standard: 10,000 images of cats → recognize cats
Meta-Learning: 5 images of new animal → recognize new animal

Approaches

MAML (Model-Agnostic Meta-Learning):
├─ Find initialization that adapts quickly
├─ Inner loop: adapt to new task
├─ Outer loop: optimize initialization
└─ Works with any gradient-based model

Prototypical Networks:
├─ Compute class prototypes (mean embeddings)
├─ Classify by nearest prototype
├─ Simple and effective
└─ Works for few-shot classification

Matching Networks:
├─ Attention-based similarity
├─ Weighted nearest neighbor
└─ Episodic training

Key Takeaways

1. Meta-learning enables few-shot learning
2. MAML finds good initialization for fast adaptation
3. Prototypical networks use class prototypes
4. Episodic training simulates few-shot scenarios
5. Applications: robotics, personalization, drug discovery
6. Transfer learning is a simpler alternative
7. Meta-learning is data-efficient for new tasks
8. Neural architecture search is related (learning to design models)