Advanced Text Summarization: Faithfulness and Controllability

Module: Natural Language Processing | Difficulty: Advanced

Faithfulness

Controllable Summarization

where = desired attributes (length, style, etc.)

Evaluation Metrics

Summarization Errors

1. Hallucination: generating unsupported information
2. Redundancy: repeating the same information
3. Incompleteness: missing key information

import torch
import torch.nn as nn

class ControllableSummarizer(nn.Module):
    def __init__(self, encoder, decoder, control_embed_dim=64):
        super().__init__()
        self.encoder = encoder
        self.decoder = decoder
        self.control_proj = nn.Linear(control_embed_dim, decoder.d_model)
    def forward(self, src, tgt, control_signal):
        enc_out = self.encoder(src)
        control_emb = self.control_proj(control_signal).unsqueeze(1)
        dec_out = self.decoder(tgt, enc_out + control_emb)
        return dec_out

Research Insight: Factual consistency is the most important aspect of summary quality, but ROUGE does not measure it. Models that optimize ROUGE can still hallucinate facts. New metrics like FactCC and SummaC specifically evaluate factual consistency.