AI tops generalist physicians at diagnostic workup

Clinical takeaway: Watch for decision-support tools that suggest a fuller line of questioning before starting tests. In the meantime, remember to be rigorous. 

Clinicians reach a diagnosis one decision at a time: what to ask next, what test to order, and whether the evidence is enough to stop. Every addition can cost money, time or delay, so the sequence can matter as much as the final call. 

Language models have scored well on licensing exams and written cases. But those formats hand over the full patient record from the start, which can overstate how a model handles a case as it unfolds. This benchmark study extends the evaluation to that setting, pitting an AI system against generalist doctors on cases that open with a few sentences and reveal the rest over time. 

An AI system, Diagnostic Orchestrator (DxO), out-diagnosed generalist physicians on both conference cases and emergency department cases. Its estimated testing costs ran higher than those of physicians, unless a budget cap was instituted to hold them down. But even with the budget cap on testing, DxO still vastly outperformed physicians. 

DxO reached 83.2% accuracy on New England Journal of Medicine clinicopathological conference cases, versus 36.1% for physicians working with reference tools, and 80.3% versus 46.8% on emergency department cases. Uncapped, DxO ordered an estimated $4,979 of testing per emergency department case versus $3,400 for physicians. On conference cases, capped DxO still reached 79.6% accuracy at an estimated $1,303 for testing, versus $4,725 for physicians. A plain GPT-5 prompt cost more still on conference cases, at $8,409 in testing costs, and reached 78.9% accuracy. 

In one sample case, questions from DxO covered symptom evolution, medications and supplements, occupational exposures, and travel. Physicians asked a few questions, ordered a test, then went back to further inquiries. Before ordering its first test on conference cases, DxO asked 25.5 questions on average as compared to 5.6 by physicians. DxO’s edge over a plain GPT-5 prompt concentrated in diagnoses that hinge on history, such as rare exposures and travel-related infections. 

DxO is a set of prompts, not a trained model. It has one language model, GPT-5 in this study, simulate a panel of five roles: one keeps a ranked list of likely diagnoses, one picks tests, one challenges the leading diagnosis, one vetoes low-yield expensive tests, and one checks that the panel’s reasoning stays consistent. The panel settles on one move at a time: ask a question, order tests, or commit to a diagnosis. Physician input and repeated testing shaped the roles. 

DxO worked all 304 New England Journal of Medicine clinicopathological conference cases and all 325 emergency department admissions from MIMIC-IV-ED, as did a plain GPT-5 prompt with no orchestration. Each case revealed information step by step as the system asked for it. Because case write-ups omit most of what a clinician would ask, a language model generated the missing answers in more than 60% of responses overall. But the authors found no sign this inflated accuracy. A GPT-5 judge graded diagnoses. Testing costs reflect 2023 list prices at one US health system. The authors, all Microsoft employees with equity, built DxO. 

Physicians in the US and UK worked from the 56 most recent conference cases: 21 without reference tools and 19 with them. Sixteen of those physicians also worked emergency department cases, with reference tools. 

The authors expect the gap to narrow against specialists or multidisciplinary teams, since conference cases are chosen for teaching value and often include specialist reflections. Physicians and DxO both worked from text alone, without images, waveforms or slides. The study measured no patient outcomes. 

“By compressing diagnosis into one inference step,” the authors noted, “static benchmarks cannot measure whether a system asks the right questions, refrains from indiscriminate testing, updates the differential diagnosis as evidence accrues, or recognizes when sufficient certainty has been achieved to conclude the workup.” 

Source: Nori H, et al. (2026 Oct 22) Science. Sequential diagnosis with language models