Clinical NLP Systems

Clinical Named Entity Recognition (NER)

from transformers import AutoTokenizer, AutoModelForTokenClassification

class ClinicalNER:
    def __init__(self, model_name="microsoft/BiomedNLP-PubMedBERT-base"):
        self.tokenizer = AutoTokenizer.from_pretrained(model_name)
        self.model = AutoModelForTokenClassification.from_pretrained(
            model_name, num_labels=12
        )
        self.label_map = {
            0: 'O', 1: 'B-DISEASE', 2: 'I-DISEASE',
            3: 'B-MEDICATION', 4: 'I-MEDICATION',
            5: 'B-PROCEDURE', 6: 'I-PROCEDURE',
            7: 'B-ANATOMY', 8: 'I-ANATOMY',
            9: 'B-SYMPTOM', 10: 'I-SYMPTOM'
        }

    def extract_entities(self, text):
        inputs = self.tokenizer(text, return_tensors="pt",
                                padding=True, truncation=True)
        logits = self.model(**inputs).logits
        predictions = torch.argmax(logits, dim=-1)[0]
        return [(self.tokenizer.decode(inputs['input_ids'][0][i]),
                 self.label_map[p.item()])
                for i, p in enumerate(predictions) if self.label_map[p.item()] != 'O']

BIO Tagging Scheme

Clinical Text Classification

Multi-Task Learning

class MultiTaskClinicalClassifier(nn.Module):
    def __init__(self, base_model, task_configs):
        super().__init__()
        self.encoder = base_model
        self.task_heads = nn.ModuleDict()
        for task_name, num_classes in task_configs.items():
            self.task_heads[task_name] = nn.Linear(
                base_model.config.hidden_size, num_classes
            )

    def forward(self, input_ids, attention_mask, task_name):
        outputs = self.encoder(input_ids=input_ids, attention_mask=attention_mask)
        return self.task_heads[task_name](outputs.last_hidden_state[:, 0, :])

Clinical Relation Extraction

Common Relations

• TREATS: Drug treats condition
• CAUSES: Condition causes symptom
• ADMINISTERED_AS: Medication route
• DOSAGE_OF: Dosage information

Pre-training on Clinical Corpora

Masked Language Modeling

Evaluation

Real-World Issue: Negation Detection Failure

Problem: A clinical NLP system extracted "pneumonia" from a note saying "No evidence of pneumonia" and flagged it as an active diagnosis. This led to unnecessary antibiotic prescriptions.

Root Cause: The NER system detected the entity but didn't understand the negation context.

Solution Applied:

1. Negation scope detection: Implemented dependency parsing to identify negation cues and their scope
2. Contextual embeddings: Used BioBERT which captures contextual meaning
3. Rule-based post-processing: Added clinical negation rules (NegEx algorithm)

Result: Negation detection accuracy improved from 72% to 94%.

Common Clinical NLP Challenges

Challenge 1: Abbreviation Resolution

Problem: "HTN" could mean hypertension, but in context "HTN" might also refer to a patient's initials.

Solution: Use context-aware abbreviation expansion. Build abbreviation dictionaries from institution-specific data.

Challenge 2: Temporal Reasoning

Problem: Distinguishing "history of MI" (past) from "rules out MI" (current differential) requires understanding temporal context.

Solution: Implement temporal taggers that identify time expressions and link them to clinical events.

Challenge 3: Inter-Annotator Disagreement

Problem: Two clinicians annotate the same note differently — one marks "mild" as a symptom, the other doesn't.

Solution: Use annotation guidelines with clear examples. Calculate inter-annotator agreement (Cohen's kappa > 0.7). Use adjudication for disagreements.

Clinical NER Performance Benchmarks

Summary

Interview Questions

Answer: The pipeline consists of Clinical Notes → Tokenization → NER Extraction → Relation Extraction → Knowledge Graph Construction. Tokenization breaks text into processable units. NER extracts entities like medications, diseases, and procedures. Relation extraction connects entities (e.g., "Drug X treats Condition Y"). Without relation extraction, you have isolated entities but no understanding of how they relate clinically—you'd know a patient has "metformin" and "diabetes" but not that metformin is treating the diabetes. This makes the knowledge graph incomplete and limits clinical decision support capabilities.

Answer: The system extracted "pneumonia" from "No evidence of pneumonia" and flagged it as active, causing unnecessary antibiotics. The solution combined dependency parsing for negation scope detection, contextual embeddings from BioBERT, and rule-based NegEx post-processing. NegEx uses trigger words ("no", "denies", "rules out") and their scope to flip entity polarity. Limitations include: it's rule-based so can't handle complex negations, struggles with double negations ("not unlikely"), and requires manually maintained trigger dictionaries. The combined approach improved accuracy from 72% to 94%.

Answer: PubMedBERT is pre-trained on biomedical literature, giving it domain-specific vocabulary and semantic understanding of medical terms. It learns representations for terms like "myocardial infarction" that generic BERT treats as unusual tokens. The 0.89 F1 on i2b2/VA (a clinical notes benchmark) indicates strong performance balancing precision and recall—catching 89% of true entities while maintaining reasonable precision. This is crucial for clinical applications where missing a disease entity (low recall) or false alarms (low precision) both have serious consequences.

Quiz