Coreference Resolution: Linking Pronouns to Entities
Module: Natural Language Processing | Difficulty: Advanced
Coreference Chain
End-to-End Model
Span Representation
Evaluation (CoNLL F1)
| Model | CoNLL F1 | Avg F1 | |-------|----------|--------| | Lee et al. (2017) | 67.2 | 64.8 | | BERT-based | 83.1 | 81.5 |
import torch
import torch.nn as nn
class CorefModel(nn.Module):
def __init__(self, bert_model, max_span_width=30):
super().__init__()
self.bert = bert_model
self.max_span_width = max_span_width
self.mention scorer = nn.Linear(768*3, 1)
self.ante scorer = nn.Linear(768*4, 1)
def get_span_repr(self, hidden, start_idx, end_idx):
start_repr = hidden[torch.arange(hidden.size(0)).unsqueeze(1), start_idx]
end_repr = hidden[torch.arange(hidden.size(0)).unsqueeze(1), end_idx]
width = end_idx - start_idx + 1
width_emb = self.width_embed(width)
return torch.cat([start_repr, end_repr, width_emb], dim=-1)
Research Insight: Coreference resolution is crucial for understanding discourse coherence. BERT-based models improved CoNLL F1 by 16 points because BERT's contextualized representations capture semantic similarity better than LSTMs.