A prospective trial shows that high accuracy cannot save emergency department AI if exhausted doctors stop clicking on it.
Clinical algorithms often perform brilliantly in simulated benchmarks, only to stall in chaotic hospital environments. A prospective evaluation analyzed SHAKED, a clinical decision support system built on multiple large language models, inside a busy tertiary emergency department over four weeks. The software delivered accurate suggestions without causing safety incidents, but physician usage collapsed as work shifts dragged on.
The DECIDE-AI stage 1 trial tracked 1,138 patients across two parallel emergency units. One unit had access to the AI tool, while the other operated under standard clinical rotations. Expert reviewers rated 99 of 100 sampled AI outputs as clinically appropriate, and researchers detected zero adverse events during the study. Yet the technology failed to alter overall patient flow in measurable ways. Length of stay remained identical at 4.9 hours in both wings (P = 0.99), and a trend toward shorter consultation cycle times of -9.4 minutes did not reach statistical significance (P = 0.077).
Accuracy fails to drive usage
The primary bottleneck was human friction rather than algorithmic error. Clinician adoption dropped steadily from 68% to 30% across the study period.
Shift fatigue directly drove this disengagement. Every additional hour on shift reduced the odds of a physician opening the software by 28% (OR = 0.72, 95% CI 0.62 to 0.83). When emergency rooms fill up, clinicians abandon non-essential software interfaces to protect their clinical speed.
These findings mirror systemic integration hurdles previously documented in research on implementing machine learning in the electronic health record. Algorithms cannot improve clinical throughput if their interface adds overhead to an overstretched workforce.
Workload dictates clinical adoption
Physicians did not reject the system entirely. Instead, they used it selectively for complex decisions where the cognitive payback was highest, showing a strong preference for AI assistance during radiology consultations (OR = 2.98, 95% CI 1.58 to 5.63).
That selective usage reveals a crucial lesson for health system leaders. While literature on evaluating the accuracy of a state-of-the-art large language model focuses primarily on diagnostic accuracy, real-world utility is bounded by human endurance and task design.
Diagnostic accuracy is a prerequisite, not a solution.
A software platform that boasts near-perfect clinical accuracy will still fail to transform patient outcomes if busy clinicians opt out during peak hours. Future randomized trials must treat sustained clinician engagement as a primary hurdle equal to diagnostic safety, because algorithmic precision alone cannot compensate for a demanding user experience.
Read the full prospective study in Nature Medicine.
