Commonsense Reasoning: Understanding the Implied

Module: Natural Language Processing | Difficulty: Advanced

Commonsense Knowledge

Benchmarks

| Benchmark | Task | Human | GPT-3 | |-----------|------|-------|-------| | PIQA | Physical | 94.9% | 79.2% | | HellaSwag | Activity | 95.6% | 78.3% | | WinoGrande | Coref | 86.7% | 77.3% |

Approaches

1. Knowledge base completion
2. Pre-trained language models
3. Neuro-symbolic methods

def commonsense_qa(model, question, choices):
    scores = []
    for choice in choices:
        prompt = f"{question} Answer: {choice}"
        inputs = tokenizer(prompt, return_tensors='pt')
        with torch.no_grad():
            outputs = model(**inputs)
            score = outputs.logits[0, -1, tokenizer.convert_tokens_to_ids(choice[0])]
        scores.append(score.item())
    return choices[scores.index(max(scores))]

Research Insight: Commonsense reasoning remains a major challenge because commonsense knowledge is vast and often implicit. Pre-trained models capture some commonsense through statistical patterns, but struggle with novel situations requiring deep understanding.