Microsoft AI Panel Outperforms Generalist Doctors in Complex Diagnosis Trials

83 percent. That is the accuracy rate achieved by Microsoft’s new diagnostic AI system, MAI-DxO, when evaluated on complex medical cases published in the New England Journal of Medicine. The figure stands in stark contrast to the 36 percent success rate recorded by primary care physicians and general internists working individually with reference materials. According to a study released this Thursday in the journal Science, the artificial intelligence system correctly identified the condition in nearly nine out of ten scenarios where human generalists missed the mark more than twice as often.

The system, known internally as DxO, functions not as a single brain but as a coordinated panel of artificial agents. These digital colleagues mimic a multidisciplinary medical team by questioning the patient, ordering necessary tests, and debating potential diagnoses before reaching a final conclusion. Mustafa Suleyman, CEO of Microsoft AI, described the result as a critical step toward achieving “medical superintelligence.” This term, defined in a concurrent manifesto, refers to the capacity to surpass the strongest contemporary human-led health systems in both medical reasoning and coordination across the full arc of care.

While the headline numbers are striking, the study includes significant caveats regarding the composition of the comparison group. The human participants were generalists, such as general internists and primary care doctors. In standard clinical practice, cases of this complexity are typically managed by specialists or collaborative physician teams, who were not included in this specific benchmark. Consequently, the results highlight the AI’s advantage over individual general practitioners rather than its performance against the full spectrum of specialized medical expertise.

The data suggests that the primary driver of improved accuracy is the power of the underlying large language models rather than the orchestration framework itself. Microsoft tested DxO using models from various providers, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest performance was observed when the system utilized OpenAI’s GPT-5 and GPT-5.5 models. However, the orchestration layer did add incremental value, particularly when paired with less capable models. For instance, Google’s Gemini 2.5 Flash saw its accuracy rise from 46.4 percent to nearly 69 percent when assisted by the DxO framework, though the gap between strong and weak models remained larger than the boost provided by the AI panel.

Cost efficiency was also a key metric in the evaluation. When running GPT-5 alone with a basic prompt, the system achieved 78.9 percent accuracy at an estimated cost of $8,400 per case. The DxO orchestration reduced that cost by approximately $3,000 while increasing accuracy by only about four points. On emergency department cases, the improvement was marginal, with little difference in either cost or accuracy between the standalone model and the orchestrated version. The benefits of the multi-agent system appear most pronounced in complex, non-emergency scenarios and when the underlying model is not at the peak of current capability.

Microsoft is positioning this technology as a solution to the growing mismatch between the demand for medical expertise and its supply. Dominic King, vice president of Health at Microsoft AI, stated that the results are substantial and exciting, noting that the core challenge in healthcare delivery today is this supply-demand imbalance. By publishing these results, Microsoft joins peers Google and OpenAI in claiming that AI can match or outperform human doctors in specific diagnostic contexts. The company emphasizes that this is not just about answering exam questions but about integrating with clinicians to improve real-world outcomes.

To validate these laboratory findings, Microsoft has initiated a partnership with Beth Israel Deaconess Medical Center in Boston. This collaboration will evaluate the clinical reasoning capabilities of DxO across diverse hospital settings, including emergency and urgent care environments. Patients will receive standard treatment, while those who consent will also engage with the AI system. Microsoft and its partners will then compare the AI’s recommendations against the decisions made by human doctors. King described this as potentially the biggest study of its kind, aiming to provide compelling evidence on the deployability of such systems at scale.

Looking ahead, Suleyman indicated that Microsoft is working with the Mayo Clinic to train a foundation model specifically for healthcare, with a release planned for next year. He believes this dedicated model will enable “really enormous leaps” in the field. Suleyman, a former co-founder of DeepMind and the startup Hark, has long advocated for AI’s role in medicine. He predicts that within 24 to 36 months, healthcare models will become an order of magnitude more capable. He envisions a 2027 scenario where AI provides “coding style breakthroughs” for medical records, allowing clinicians to stream highly accurate predictions of how a patient’s condition might unfold, a foresight he describes as “gold dust” for medical practitioners.

The broader implication is a shift in how Microsoft approaches its AI portfolio. The company already sells dictation and radiology tools to clinicians and views DxO as a natural expansion of those efforts. Suleyman stated that Microsoft is committed to having the “absolute number one state-of-the-art medical clinical decision support products” in the coming years. He argues that this focus on deep integration with healthcare systems could serve as a differentiator for Microsoft compared to other frontier AI labs. As the company moves from benchmarking individual tasks to integrating with real-world clinical workflows, the next 24 to 36 months are expected to be pivotal in determining whether these theoretical advantages translate into tangible improvements in patient care.