Article · AI and medicine

Did AI diagnose hard cases better than doctors?

A calm read of Microsoft's new MAI-DxO study: how it worked, the exact numbers, what most headlines left out, and what comes next.

Mohamed Maslooh · 2026-11-01 · 6 min read

The Microsoft sign at the company's campus. Image: Unknown, Public domain, Wikimedia Commons.

The news in two lines

On 22 October 2026 Microsoft published MAI-DxO in Science. It reached the right diagnosis in 83% of some of the hardest published medical cases; the doctors it was compared with got 36%.

The number spread fast. As with any study, the details tell the full story.

Mustafa Suleyman, CEO of Microsoft AI, called the result a step toward "medical superintelligence". Image: Joi Ito from Cambridge, MA, USA, CC BY 2.0, Wikimedia Commons.

Not one AI, a whole panel

MAI-DxO is not a single model answering a question. It is an orchestrator that runs several AI agents together, like a panel of doctors on one case, each with a job: one proposes likely diagnoses, one picks the test that separates them, one challenges the others' reasoning, one watches the cost, one checks nothing was missed. They debate at every step until they agree.

The test: 304 cases, one step at a time

The cases come from the hardest cases published in the New England Journal of Medicine.

The system did not get the whole case at once. It started from a short summary, then asked questions and ordered tests and scans one by one, and every test had a price, as with a real doctor and a real patient.

A normal chest X-ray. The system ordered tests and scans one at a time, each with a price. Image: Mikael Häggström, CC0, Wikimedia Commons.

The result

Accuracy on 304 hard cases
	Correct diagnosis
MAI-DxO	83.2%
Generalist doctors, with books and the internet	36.1%
The same doctors, without them	22.3%

The system was right in 83.2% of cases. The doctors (generalists: internal medicine and general practice) were right in 36.1% with books and the internet, and 22.3% without them.

And in the emergency department?

They also tested it on 325 past emergency admissions, and the gap stayed large.

Emergency department: 325 admissions
	Correct diagnosis
MAI-DxO	80.3%
Doctors	46.8%

An emergency department entrance at a US hospital. Image: Harrison Keely, CC BY 4.0, Wikimedia Commons.

And the cost?

The version that scored 83% had no budget and ordered about $5,431 of tests per case, a bit more than the doctors' estimate ($3,049 to $4,725).

With a budget it stayed near 80% for much less: 80.9% at $2,241, and 79.6% at $1,303.

Accuracy against cost per case
	Accuracy	Cost per case
MAI-DxO, no budget	83.2%	$5,431
MAI-DxO, with a budget	80.9%	$2,241
MAI-DxO, lower budget	79.6%	$1,303
Doctors, estimate	36.1%	$3,049 to $4,725

Cost is the estimated price of tests and visits, in US dollars. Up and to the left is better: more accurate, cheaper.

These are estimated test and visit costs inside a simulation, not real bills.

The panel or the model?

The study tried models from OpenAI, Google, Anthropic, Meta and DeepSeek. The panel lifted every model, most of all the weaker ones.

The same model, alone and inside the panel
	the model alone	inside the panel (MAI-DxO)
GPT-5.5	72%	≈87%
GPT-5	78.9%	≈83%
Gemini 2.5 Flash	46.4%	≈69%

≈ means about. Higher is better.

Much of the result comes from the model itself, and the panel adds to it. In emergency cases the panel added almost nothing on top of the strongest model.

Before we get excited

The comparison was with generalists, not specialists or a full team, who are the people who actually handle cases like these.
Every case was a past record. No patient has been treated with this system yet; the result comes from a simulation.
Published cases are chosen because they are hard and rare, which is not what a normal hospital day looks like.

What comes next

Microsoft will test the system with real patients in emergency and urgent care at Beth Israel Deaconess Medical Center in Boston: patients get standard care, and for those who agree, the system's recommendations are compared with their doctors'. It is also working with Mayo Clinic on a medical foundation model planned for next year.

That result is the one that will matter.

Beth Israel Deaconess Medical Center in Boston, where the system will be tested with real patients. Image: Tim Pierce, CC BY 3.0, Wikimedia Commons.

The Gonda Building at Mayo Clinic, Microsoft's partner on a medical foundation model. Image: Nephron, CC BY-SA 3.0, Wikimedia Commons.

What do you think?

Could AI replace doctors? Tell me in the comments on the post, or message me.

Sources

The study was published in Science on 22 October 2026.

All articles