AI falls short on differential diagnosis, despite high accuracy rates

Overview

Mass General Brigham researchers published findings showing that while large language models (LLMs) demonstrate high diagnostic accuracy rates, they fail to perform the clinical reasoning required for differential diagnosis—the systematic process of distinguishing between conditions with similar presentations. This gap matters for healthcare practices increasingly deploying AI tools for clinical decision support, documentation, and patient communication. The study highlights that accuracy alone does not equal clinical competence, raising concerns about liability exposure when practices rely on AI systems that cannot properly navigate diagnostic uncertainty or explain their reasoning process.

Technical Details

The Mass General Brigham study examined how LLMs handle differential diagnosis scenarios requiring:

• Sequential reasoning through multiple possible conditions
• Evaluation of competing hypotheses based on clinical presentation
• Identification of discriminating features between similar diagnoses
• Documentation of decision-making rationale for the medical record

While the models achieved high accuracy on straightforward diagnostic tasks, they struggled with complex cases requiring nuanced clinical judgment. The models could not reliably explain why they excluded certain diagnoses or how they weighted conflicting evidence—the foundational reasoning documented in clinical notes and essential for defending medical decisions in malpractice or regulatory review.

Practical Implications

For independent practices, this creates several compliance and operational risks:

• Documentation gaps: AI-generated clinical notes may lack the reasoning trail required for medical record integrity under 45 CFR §164.312(d)
• Liability exposure: Providers remain legally responsible for AI-assisted clinical decisions, but cannot defend decisions they don't understand
• Patient safety risks: Diagnostic errors are the leading cause of medical malpractice claims; relying on opaque AI reasoning increases this exposure
• Training requirements: Staff using AI clinical tools need documented competency training—but how do you train on tools that can't explain their reasoning?

What This Means for Your Practice

If your practice uses or is considering AI tools for clinical documentation, decision support, or patient triage:

Immediate actions:

• Audit AI-generated content in patient records for clinical reasoning documentation—does it show differential diagnosis thinking or just conclusions?
• Review vendor BAAs for AI tool providers—do they specify data use for model training? What happens to PHI used in prompts?
• Document physician oversight of AI-generated clinical content—who reviews, what gets changed, how is review tracked?
• Update training protocols—staff need documented training on AI tool limitations, not just how to use the interface

Strategic considerations:

• AI tools are not set-and-forget—they require active governance, monitoring, and documentation of oversight
• Diagnostic accuracy ≠ clinical reasoning—your medical records must show the reasoning behind decisions, not just the conclusions
• You own the liability—vendors disclaim responsibility for clinical decisions; providers bear full legal and regulatory accountability