Text Summarization: Abstractive and Extractive Methods
Module: Natural Language Processing | Difficulty: Advanced
Abstractive Summarization
Coverage Mechanism
Extractive Summarization
ROUGE Scores
| Metric | Precision | Recall | F1 |
|---|---|---|---|
| ROUGE-1 | 45.2 | 52.3 | 48.5 |
| ROUGE-2 | 18.3 | 21.5 | 19.7 |
| ROUGE-L | 41.7 | 48.9 | 45.0 |
import torch
import torch.nn as nn
class SummarizationModel(nn.Module):
def __init__(self, encoder, decoder, vocab_size):
super().__init__()
self.encoder = encoder
self.decoder = decoder
self.generator = nn.Linear(decoder.d_model, vocab_size)
def forward(self, src, tgt, coverage=None):
enc_out = self.encoder(src)
dec_out = self.decoder(tgt, enc_out, coverage)
logits = self.generator(dec_out)
return logits
Research Insight: Coverage mechanism prevents repeated attention to the same positions, addressing the repetition problem in abstractive summarization. The key insight is that summarization requires understanding both what to include and what to omit.