Clinical NLP Systems

Clinical Named Entity Recognition (NER)

from transformers import AutoTokenizer, AutoModelForTokenClassification

class ClinicalNER:
    def __init__(self, model_name="microsoft/BiomedNLP-PubMedBERT-base"):
        self.tokenizer = AutoTokenizer.from_pretrained(model_name)
        self.model = AutoModelForTokenClassification.from_pretrained(
            model_name, num_labels=12
        )
        self.label_map = {
            0: 'O', 1: 'B-DISEASE', 2: 'I-DISEASE',
            3: 'B-MEDICATION', 4: 'I-MEDICATION',
            5: 'B-PROCEDURE', 6: 'I-PROCEDURE',
            7: 'B-ANATOMY', 8: 'I-ANATOMY',
            9: 'B-SYMPTOM', 10: 'I-SYMPTOM'
        }

    def extract_entities(self, text):
        inputs = self.tokenizer(text, return_tensors="pt",
                                padding=True, truncation=True)
        logits = self.model(**inputs).logits
        predictions = torch.argmax(logits, dim=-1)[0]
        return [(self.tokenizer.decode(inputs['input_ids'][0][i]),
                 self.label_map[p.item()])
                for i, p in enumerate(predictions) if self.label_map[p.item()] != 'O']

BIO Tagging Scheme

Clinical Text Classification

Multi-Task Learning

class MultiTaskClinicalClassifier(nn.Module):
    def __init__(self, base_model, task_configs):
        super().__init__()
        self.encoder = base_model
        self.task_heads = nn.ModuleDict()
        for task_name, num_classes in task_configs.items():
            self.task_heads[task_name] = nn.Linear(
                base_model.config.hidden_size, num_classes
            )

    def forward(self, input_ids, attention_mask, task_name):
        outputs = self.encoder(input_ids=input_ids, attention_mask=attention_mask)
        return self.task_heads[task_name](outputs.last_hidden_state[:, 0, :])

Clinical Relation Extraction

Common Relations

• TREATS: Drug treats condition
• CAUSES: Condition causes symptom
• ADMINISTERED_AS: Medication route
• DOSAGE_OF: Dosage information

Pre-training on Clinical Corpora

Masked Language Modeling

Evaluation

Real-World Issue: Negation Detection Failure

Problem: A clinical NLP system extracted "pneumonia" from a note saying "No evidence of pneumonia" and flagged it as an active diagnosis. This led to unnecessary antibiotic prescriptions.

Root Cause: The NER system detected the entity but didn't understand the negation context.

Solution Applied:

1. Negation scope detection: Implemented dependency parsing to identify negation cues and their scope
2. Contextual embeddings: Used BioBERT which captures contextual meaning
3. Rule-based post-processing: Added clinical negation rules (NegEx algorithm)

Result: Negation detection accuracy improved from 72% to 94%.

Common Clinical NLP Challenges

Challenge 1: Abbreviation Resolution

Problem: "HTN" could mean hypertension, but in context "HTN" might also refer to a patient's initials.

Solution: Use context-aware abbreviation expansion. Build abbreviation dictionaries from institution-specific data.

Challenge 2: Temporal Reasoning

Problem: Distinguishing "history of MI" (past) from "rules out MI" (current differential) requires understanding temporal context.

Solution: Implement temporal taggers that identify time expressions and link them to clinical events.

Challenge 3: Inter-Annotator Disagreement

Problem: Two clinicians annotate the same note differently — one marks "mild" as a symptom, the other doesn't.

Solution: Use annotation guidelines with clear examples. Calculate inter-annotator agreement (Cohen's kappa > 0.7). Use adjudication for disagreements.

Clinical NER Performance Benchmarks

Summary