Microsoft: Our Medical AI Outperforms Doctors in Complex Cases

Microsoft has announced the results of a study published in the journal Science indicating that its AI-based diagnostic system was able to analyze highly complex medical cases with an accuracy more than double that achieved by general physicians in a simulated environment. Mustafa Suleyman, CEO of the company's AI division, described the result as a step toward what he calls "medical superintelligence" — a long-term goal he has been pursuing for years.

How does the MAI-DxO system work?

The system, known in short as DxO, does not rely on a single model answering a question. Instead, it coordinates a group of intelligent agents that behave like an advisory panel of physicians. This "digital team" asks the patient questions, orders the necessary tests and examinations, then deliberates internally to reach a final diagnosis, while factoring in the estimated cost of each step.

In the test, complex cases drawn from the New England Journal of Medicine were used, and the system recorded correct answers in 83% of cases, compared to just 36% for primary care and general internal medicine physicians who worked individually with the ability to consult references. However, one essential caveat is worth noting: such cases are usually managed by specialist physicians or full medical teams, and these were not part of the comparison.

Who deserves the real credit: the orchestration or the model?

The study's own figures reveal that the greater part of the improvement comes from the strength of the underlying models, while the orchestration layer built by Microsoft adds only limited additional gains in certain contexts. The system was tested with models from OpenAI, Google, Anthropic, Meta, and DeepSeek, with the best results coming from GPT-5 and GPT-5.5.

• GPT-5 on its own, with simple prompting, scored 78.9% on complex cases at an estimated cost of around $8,400 per case; DxO reduced the cost by roughly $3,000 but raised accuracy by only four points.
• With GPT-5.5, accuracy on complex cases rose from 72% to nearly 87%, at a slightly lower cost.
• In emergency department cases, the difference almost vanished: around 85% with or without the system, at a comparable cost.
• The greatest benefit appeared with less capable models, such as Gemini 2.5 Flash, which went from 46.4% to around 69%.

The company's researchers explain this by noting that margins for improvement naturally shrink as a model approaches the theoretical performance ceiling, and that the role of the orchestration layer is focused on balancing accuracy against cost rather than on multiplying accuracy alone.

The supply-and-demand gap in medical expertise

Dominic King, Vice President for Health at Microsoft's AI division, believes the biggest challenge facing healthcare systems today is the widening gap between demand for medical expertise and its supply — something these tools aim to address as a layer supporting the physician's decision rather than replacing it.

From simulation to the hospital

The next step is testing the system in real clinical settings through a partnership with Beth Israel Deaconess Medical Center in Boston. Under the plan, patients receive their full standard care — triage, tests, consultations, and the discharge decision — and, in parallel, those who consent interact with the AI system, after which its recommendations are compared with the physicians' decisions. King describes this trial as potentially the largest of its kind, as it covers both emergency departments and urgent care settings. It is worth noting that Google published a clinical study this month with the same center in The Lancet evaluating its conversational diagnostic system AMIE in urgent care visits.

A healthcare foundation model on the way

Suleyman revealed that Microsoft is working with the Mayo Clinic to train a foundation model dedicated to healthcare, expected to launch next year, noting that DxO represents part of the infrastructure needed to achieve genuine breakthroughs. He expects health models to become a full degree better in terms of capability within 24 to 36 months, pointing to what he calls the "flywheel": a model in actual production that improves gradually through real-world use, with gaps identified and addressed — as previously happened with coding and chat tools.

Suleyman goes further, suggesting that 2027 may witness a shift in how medical records are handled, similar to what happened in software development, so that models become capable of generating accurate predictions about the trajectory of a patient's condition — a forward-looking capability he considers of great value to the physician.

A balanced reading

The results are intriguing, but they remain within the realm of simulation rather than clinical practice, and the comparison was made against general physicians working alone on cases that normally call for specialist teams. The real value will become clear through the field trials now under way, and through the ability of these systems to operate within the hospital context with all its complexity and variety of cases.

✦ بقلم فريق دروب أيديا