πŸŽ‰ 75% of content is free forever β€” Unlock Premium from $10/mo β†’
CW
Search courses…
πŸ’Ό Servicesℹ️ Aboutβœ‰οΈ ContactView Pricing Plansfrom $10

PubMed Research Agent

AI AgentsMedical Research Agent🟒 Free Lesson

Advertisement

PubMed Research Agent

Medical Research AgentPubMed APIPaper ParserSummarizerCitation MgrEvidence SynthesizerReport GeneratorResearch Orchestrator

What is a Medical Research Agent?

Medical research agents automate literature review by searching PubMed, extracting key findings, summarizing papers, and synthesizing evidence across studies. They help researchers quickly understand the state of knowledge on a medical topic.

The key capabilities are: automated search with MeSH terms, structured extraction of study design and outcomes, critical appraisal of evidence quality, and synthesis of findings across multiple papers with proper citations.

These agents accelerate evidence-based medicine by reducing the time needed for systematic reviews and keeping clinicians updated on relevant research.

Project Overview

We will build a PubMed research agent that:

  • Searches PubMed using the E-utilities API
  • Parses paper metadata and abstracts
  • Extracts study design, sample size, and outcomes
  • Summarizes key findings with evidence levels
  • Manages citations in standard formats
  • Generates literature review reports

Expected outcome: An agent that performs automated literature reviews from natural language queries.

Difficulty: Advanced (requires understanding of medical informatics, evidence levels, and citation formats)

Tools & Setup

ToolVersionPurpose
Python3.11+Core language
biopython1.80+PubMed E-utilities
requests2.31+HTTP client
openai1.0+LLM backbone
pydantic2.0+Data models

Step 1: Environment Setup

python -m venv venv
source venv/bin/activate
pip install biopython requests openai pydantic

Step 2: Project Structure

Architecture Diagram
medical-agent/
β”œβ”€β”€ pubmed/
β”‚   β”œβ”€β”€ __init__.py
β”‚   └── client.py
β”œβ”€β”€ processing/
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ paper_parser.py
β”‚   └── summarizer.py
β”œβ”€β”€ synthesis/
β”‚   β”œβ”€β”€ __init__.py
β”‚   └── evidence_synthesizer.py
β”œβ”€β”€ agent.py
└── main.py

Step 3: PubMed Client

# pubmed/client.py
from Bio import Entrez
from typing import List, Dict, Optional
import time

Entrez.email = "research-agent@example.com"

class PubMedClient:
    def __init__(self, api_key: Optional[str] = None):
        if api_key:
            Entrez.api_key = api_key

    def search(
        self,
        query: str,
        max_results: int = 20,
        sort: str = "relevance",
    ) -> List[str]:
        handle = Entrez.esearch(
            db="pubmed",
            term=query,
            retmax=max_results,
            sort=sort,
            usehistory="y",
        )
        results = Entrez.read(handle)
        handle.close()
        return results.get("IdList", [])

    def fetch_papers(self, pmids: List[str]) -> List[Dict]:
        if not pmids:
            return []
        handle = Entrez.efetch(
            db="pubmed",
            id=",".join(pmids),
            rettype="xml",
            retmode="xml",
        )
        records = Entrez.read(handle)
        handle.close()
        papers = []
        for article in records.get("PubmedArticle", []):
            paper = self._parse_article(article)
            papers.append(paper)
        return papers

    def _parse_article(self, article: dict) -> Dict:
        medline = article.get("MedlineCitation", {})
        article_data = medline.get("Article", {})
        title = article_data.get("ArticleTitle", "No title")
        abstract_parts = article_data.get("Abstract", {}).get("AbstractText", [])
        abstract = " ".join(str(p) for p in abstract_parts)
        authors = []
        for author in article_data.get("AuthorList", []):
            last = author.get("LastName", "")
            first = author.get("ForeName", "")
            if last:
                authors.append(f"{last}, {first}")
        journal = article_data.get("Journal", {}).get("Title", "")
        pub_date = article_data.get("Journal", {}).get("JournalIssue", {}).get("PubDate", {})
        year = pub_date.get("Year", "")
        month = pub_date.get("Month", "")
        pmid = str(medline.get("PMID", ""))
        keywords = [kw.strip() for kw in medline.get("KeywordList", [[]])[0]] if medline.get("KeywordList") else []
        return {
            "pmid": pmid,
            "title": title,
            "abstract": abstract,
            "authors": authors,
            "journal": journal,
            "year": year,
            "month": month,
            "keywords": keywords,
            "doi": self._extract_doi(article),
        }

    def _extract_doi(self, article: dict) -> str:
        for eid in article.get("PubmedData", {}).get("ArticleIdList", []):
            if str(eid.attributes.get("IdType", "")) == "doi":
                return str(eid)
        return ""

    def search_and_fetch(
        self, query: str, max_results: int = 10
    ) -> List[Dict]:
        pmids = self.search(query, max_results)
        time.sleep(0.5)
        return self.fetch_papers(pmids)

Step 4: Paper Parser and Summarizer

# processing/paper_parser.py
from openai import OpenAI
from typing import Dict
import json

class PaperParser:
    def __init__(self, model: str = "gpt-4-turbo-preview"):
        self.client = OpenAI()
        self.model = model

    def extract_structure(self, paper: Dict) -> Dict:
        response = self.client.chat.completions.create(
            model=self.model,
            messages=[
                {"role": "system", "content": """Extract structured information from this medical paper.
                Return JSON:
                {
                    "study_design": "RCT|cohort|case-control|cross-sectional|case-report|review|meta-analysis",
                    "sample_size": number or null,
                    "population": "description",
                    "intervention": "description or null",
                    "outcomes": ["list of measured outcomes"],
                    "key_findings": ["list of main findings"],
                    "limitations": ["list of limitations"],
                    "evidence_level": "1a|1b|2a|2b|3|4|5"
                }"""},
                {"role": "user", "content": f"Title: {paper['title']}\n\nAbstract: {paper['abstract']}"},
            ],
            temperature=0.0,
        )
        try:
            return json.loads(response.choices[0].message.content)
        except:
            return {"study_design": "unknown", "evidence_level": "5"}

# processing/summarizer.py
from openai import OpenAI
from typing import Dict, List

class PaperSummarizer:
    def __init__(self, model: str = "gpt-4-turbo-preview"):
        self.client = OpenAI()
        self.model = model

    def summarize_paper(self, paper: Dict) -> str:
        response = self.client.chat.completions.create(
            model=self.model,
            messages=[
                {"role": "system", "content": """Summarize this medical paper for a clinician.
                Include: objective, methods, key findings, clinical implications.
                Be concise (150-200 words)."""},
                {"role": "user", "content": f"Title: {paper['title']}\nAuthors: {', '.join(paper['authors'][:3])}\nJournal: {paper['journal']} ({paper['year']})\n\n{paper['abstract']}"},
            ],
            temperature=0.2,
        )
        return response.choices[0].message.content

    def synthesize_evidence(
        self, papers: List[Dict], question: str
    ) -> str:
        summaries = []
        for p in papers[:10]:
            summaries.append(f"- {p['title']} ({p['year']})\n  {p.get('abstract', '')[:300]}")
        papers_text = "\n\n".join(summaries)
        response = self.client.chat.completions.create(
            model=self.model,
            messages=[
                {"role": "system", "content": """Synthesize evidence from multiple studies.
                Provide: overall conclusion, consistency of findings, evidence gaps.
                Use GRADE-like language for evidence certainty."""},
                {"role": "user", "content": f"Clinical Question: {question}\n\nStudies:\n{papers_text}\n\nSynthesize the evidence:"},
            ],
            temperature=0.3,
        )
        return response.choices[0].message.content

    def format_citation(self, paper: Dict, style: str = "AMA") -> str:
        authors = paper["authors"]
        if len(authors) > 3:
            author_str = f"{authors[0]}, et al."
        else:
            author_str = ", ".join(authors)
        if style == "AMA":
            return f"{author_str}. {paper['title']}. {paper['journal']}. {paper['year']};{paper.get('doi', '')}"
        elif style == "APA":
            return f"{author_str} ({paper['year']}). {paper['title']}. {paper['journal']}. {paper.get('doi', '')}"
        return f"{author_str}. {paper['title']}. {paper['journal']}. {paper['year']}"

Step 5: Complete Agent

# agent.py
from pubmed.client import PubMedClient
from processing.paper_parser import PaperParser
from processing.summarizer import PaperSummarizer
from typing import Dict, List

class MedicalResearchAgent:
    def __init__(self, model: str = "gpt-4-turbo-preview", api_key: str = None):
        self.pubmed = PubMedClient(api_key=api_key)
        self.parser = PaperParser(model)
        self.summarizer = PaperSummarizer(model)

    def research(self, question: str, max_papers: int = 15) -> Dict:
        papers = self.pubmed.search_and_fetch(question, max_papers)
        analyzed = []
        for paper in papers:
            structure = self.parser.extract_structure(paper)
            summary = self.summarizer.summarize_paper(paper)
            analyzed.append({
                **paper,
                "structure": structure,
                "summary": summary,
                "citation_ama": self.summarizer.format_citation(paper, "AMA"),
            })
        synthesis = self.summarizer.synthesize_evidence(analyzed, question)
        return {
            "question": question,
            "total_papers": len(analyzed),
            "papers": analyzed,
            "synthesis": synthesis,
            "evidence_levels": [p["structure"].get("evidence_level", "5") for p in analyzed],
        }

    def get_paper_details(self, pmid: str) -> Dict:
        papers = self.pubmed.fetch_papers([pmid])
        if not papers:
            return {"error": "Paper not found"}
        paper = papers[0]
        structure = self.parser.extract_structure(paper)
        summary = self.summarizer.summarize_paper(paper)
        return {**paper, "structure": structure, "summary": summary}

Mathematical Foundation

Evidence Level Hierarchy:

Intuition: Higher levels (1a > 5) represent more reliable evidence with lower bias risk.

Heterogeneity Score:

Where is Cochran's Q statistic and is degrees of freedom.

Intuition: Measures inconsistency across studies. indicates substantial heterogeneity.

Testing & Evaluation

import pytest
from pubmed.client import PubMedClient

def test_search():
    client = PubMedClient()
    pmids = client.search("COVID-19 vaccine efficacy", max_results=5)
    assert len(pmids) > 0

def test_fetch():
    client = PubMedClient()
    papers = client.fetch_papers(["33780960"])
    assert len(papers) == 1
    assert "title" in papers[0]

Performance Metrics

MetricValueNotes
PubMed Search Time1-2sE-utilities API
Paper Fetch Time1-3sBatch of 10 papers
Parsing Time2-5sPer paper
Summarization Time3-8sPer paper
Evidence Synthesis10-30s10+ papers

Deployment

# main.py
from agent import MedicalResearchAgent
import json

def main():
    agent = MedicalResearchAgent()
    question = input("Research question: ").strip()
    result = agent.research(question)
    print(f"\nFound {result['total_papers']} papers\n")
    print(f"Evidence Synthesis:\n{result['synthesis']}\n")
    for p in result["papers"][:5]:
        print(f"\n{p['citation_ama']}\n{p['summary']}")

if __name__ == "__main__":
    main()

Real-World Use Cases

  • Clinical Decision Support: Quick evidence lookup at point of care
  • Systematic Reviews: Accelerate literature review process
  • Drug Research: Track efficacy and safety data
  • Grant Writing: Literature review for proposals
  • Continuing Education: Stay updated on latest evidence

Common Pitfalls & Solutions

PitfallSolution
API rate limitsImplement delays between requests
Abstract-only biasNote when full text unavailable
Language biasFocus on English papers, note exclusion
Publication biasSearch trial registries for unpublished data
MeSH term variationUse multiple search terms and synonyms

Summary with Key Takeaways

  • PubMed E-utilities provides programmatic access to 35M+ medical papers
  • Structured extraction enables quantitative evidence synthesis
  • Evidence level classification helps assess study quality
  • Automated summarization saves hours of manual review
  • Always note limitations including abstract-only analysis and language bias

Need Expert AI Agents Help?

Get personalized tutoring, project support, or professional consulting.

Advertisement