Word Sense Disambiguation: Context and Ambiguity

Module: Natural Language Processing | Difficulty: Advanced

WSD Task

Contextual Approach

Knowledge-Based

Results

| Model | WSD Accuracy | |-------|--------------| | Most Frequent | 66.5% | | BERT | 78.1% | | RoBERTa | 82.3% | | Gloss-Enriched | 85.6% |

import torch
import torch.nn as nn

class WSDModel(nn.Module):
    def __init__(self, bert_model, n_senses):
        super().__init__()
        self.bert = bert_model
        self.sense_classifier = nn.Linear(768, n_senses)
    def forward(self, input_ids, attention_mask, word_mask):
        outputs = self.bert(input_ids, attention_mask=attention_mask)
        hidden = outputs.last_hidden_state
        word_repr = hidden[word_mask].mean(dim=1)
        return self.sense_classifier(word_repr)

Research Insight: Contextual embeddings dramatically improved WSD because they capture word meaning in context. The key insight is that word senses are not discrete categories but continuous distributions. Recent models use gloss information to improve disambiguation.