Microsoft says its AI beat doctors on hard diagnoses

By VK

Published 25 Oct 2026, 11:45 UTC

Reporting from The Verge, TechCrunch

Microsoft says its MAI-DxO system scored 83% on hard NEJM cases in a Science study, against 36% for generalist doctors using references.

What it means for founders

Orchestration alone is a thin moat. If your health product wraps frontier models in an agent workflow, note how unevenly the orchestrator helped: about 15 points in one setup, almost none in others. Data, clinical validation and distribution likely matter more.
Expect buyers to ask for cost per case. SDBench scores workups on accuracy and test spend. Running it on real cases needs an NEJM license and PhysioNet credentialing, and payers already question AI-driven costs, as Enki reported when insurers said hospitals' AI billing tools added $942 million.
An incumbent is moving in. TechCrunch reports Microsoft sees DxO building on the dictation and radiology software it already sells clinicians, so decision support startups face a rival with a hospital foothold.
Watch the Beth Israel study. Its start date and first results are the signal, set against the Lancet study Google ran there on its AMIE diagnostic system.

The story

Microsoft AI says its diagnostic system, MAI-DxO, beat physicians on hard medical cases in a study published in Science on Thursday, October 22. Chief executive Mustafa Suleyman frames it as progress toward what Microsoft calls medical superintelligence. The figures are Microsoft's own, from a benchmark it built from past cases; no live patients were involved.

How Microsoft ran the test

The company's announcement describes SDBench, a benchmark of 304 complex New England Journal of Medicine case records plus routine emergency department cases from the MIMIC-IV-ED database. Each case opens with a summary hiding the answer; doctor or AI then asks questions and orders tests step by step. According to the SDBench project page, a gatekeeper answers requests from the case text, inventing realistic results the record lacks, and an automated judge grades diagnoses on a 1 to 5 scale, counting 4 or 5 as correct.

MAI-DxO runs several AI agents as a virtual panel that debate diagnoses and tests while weighing their estimated cost.

The numbers, and what drives them

On the NEJM cases Microsoft reports 83% accuracy for MAI-DxO, against 36% for physicians allowed textbooks and the internet and 22% for those working without them. On emergency cases, 80% against 47%. TechCrunch reported that the NEJM doctors were primary care physicians and general internists working solo, not specialists or the multi-doctor teams that usually take such cases.

TechCrunch's breakdown credits the underlying model with much of the result. Given only a basic prompt, GPT-5 hit 78.9% on hard cases at an estimated $8,400 each; the orchestrator trimmed roughly $3,000 and added about four points. With GPT-5.5, accuracy on hard cases rose from 72% to nearly 87%, yet on emergency cases both setups landed near 85%. Microsoft says its gains held with newer models, though The Verge noted the paper omits the latest frontier releases.

What comes next

Microsoft plans a prospective study with Boston's Beth Israel Deaconess Medical Center: consenting emergency and urgent care patients will use the system alongside normal care, and its calls will be compared with doctors'. Health lead Dominic King told The Verge he expects a start within a couple of months. "It needs clinical validation," Suleyman told The Verge. He told TechCrunch a healthcare model being trained with Mayo Clinic is planned for next year. SDBench code is public.