Cross-Lingual Transfer: Zero-Shot Across Languages
Module: Natural Language Processing | Difficulty: Advanced
Cross-Lingual Benchmark
| Task | Languages | Best Model |
|---|---|---|
| XNLI | 15 | XLM-R |
| MLQA | 7 | mBERT |
| XQuAD | 11 | XLM-R |
Transfer Learning
Language-Agnostic Representation
import torch
import torch.nn as nn
class CrossLingualClassifier(nn.Module):
def __init__(self, xlm_roberta, n_classes):
super().__init__()
self.encoder = xlm_roberta
self.classifier = nn.Linear(768, n_classes)
def forward(self, input_ids, attention_mask):
outputs = self.encoder(input_ids, attention_mask=attention_mask)
cls_output = outputs.last_hidden_state[:, 0]
return self.classifier(cls_output)
Research Insight: Cross-lingual transfer works best when the source and target languages are typologically similar. Language-agnostic representations enable zero-shot transfer, but performance degrades for distant language pairs.