A specialized neural network outperformed both emergency physicians and general-purpose chatbots in detecting life-threatening heart blockages on electrocardiograms.

When a patient enters the emergency room with a blocked coronary artery, minutes decide whether heart muscle lives or dies. For decades, doctors relied on rigid, traditional ECG patterns to spot these emergencies. But the clinical paradigm is shifting to occlusion myocardial infarction (OMI), a broader, more subtle category of blockages that traditional rules often miss.

The tech industry wants us to believe that massive, general-purpose large language models can handle any diagnostic task. This study challenges that assumption. It proves that when stakes are highest, task-specific clinical AI beats generalist giants.

The head-to-head test

Researchers evaluated diagnostic accuracy using 36 twelve-lead ECGs from patients referred for emergent coronary angiography. The cohort included 24 patients with angiographically confirmed acute coronary occlusion and 12 without. Five emergency medicine specialists, five residents, ChatGPT 5.2, Gemini 3 Pro, and a dedicated deep neural network called Queen of Hearts (QoH) all analyzed the same cases.

The specialized AI dominated the field, while generalist models struggled to provide safe, consistent answers.

The performance gap

• The Queen of Hearts model achieved an area under the curve (AUC) of 0.96, with 95.8% sensitivity, missing only one of the 24 occlusions.
• Emergency specialists achieved 75.0% accuracy, with 66.7% sensitivity and 91.7% specificity.
• ChatGPT 5.2 reached an AUC of 0.81 with high specificity of 91.7% but a low sensitivity of 54.2%.
• Gemini 3 Pro performed worst, with an AUC of 0.62 and an overall accuracy of just 55.6%.

Why generalists fail

The real danger of general-purpose models in clinical settings is their inconsistency. ChatGPT and Gemini produced highly variable interpretations across five repeated sessions, yielding low reliability scores with a Fleiss’ kappa of 0.24 to 0.49. Worse, Gemini displayed severe overconfidence, marked by a poor Brier score of 0.40.

This is a critical warning for hospital administrators tempted by cheap, all-in-one AI platforms. A chatbot that changes its diagnosis across different sessions is a liability in acute care. While human specialists achieved high specificity, their 66.7% sensitivity means they missed a third of the active blockages. The specialized AI bridged this gap safely, achieving 86.1% overall accuracy.

We must acknowledge the study’s limitations, particularly its small, retrospective sample of 36 ECGs. It needs validation in larger, real-time clinical trials before widespread deployment. However, the stark contrast in reliability suggests that generalist models should remain strictly supervised assistants, while specialized, deterministic neural networks take the lead in clinical decision support.

Read the full study in BMC Emergency Medicine.