Document Ranking: Learning to Rank for Information Retrieval

Module: Natural Language Processing | Difficulty: Advanced

BM25

Learning to Rank

Neural Ranking

Evaluation

| Metric | BM25 | BERT | ColBERT | |--------|------|------|---------| | NDCG@10 | 0.45 | 0.52 | 0.55 | | MAP | 0.42 | 0.49 | 0.53 |

import torch
import torch.nn as nn

class CrossEncoderRanker(nn.Module):
    def __init__(self, bert_model):
        super().__init__()
        self.bert = bert_model
        self.classifier = nn.Linear(768, 1)
    def forward(self, input_ids, attention_mask):
        outputs = self.bert(input_ids, attention_mask=attention_mask)
        cls_output = outputs.last_hidden_state[:, 0]
        return self.classifier(cls_output).squeeze(-1)

Research Insight: Bi-encoders are faster but less accurate than cross-encoders. ColBERT combines the benefits of both by using late interaction, achieving cross-encoder quality with bi-encoder efficiency.