Algorithms are spotting conditions faster in primary care, but they are not making patients any healthier.
High diagnostic accuracy on paper does not guarantee clinical benefit in the exam room. A systematic review published in the Journal of Medical Internet Research shows that while AI clinical decision support tools can flag risks, zero trials proved they improve patient outcomes.
That disconnect is the real story.
For years health systems have treated diagnostic performance as a proxy for better care. This review confirms that spotting a disease earlier only helps if it leads to effective, safe interventions.
Spotty real-world detection
Out of 66 full-text reports screened from 2011 through 2025, researchers identified 10 appraised AI-enabled studies in routine ambulatory care. Detection results varied wildly across conditions. An AI tool for electrocardiography boosted new low ejection fraction diagnoses with an odds ratio of 1.32 (95% CI 1.01-1.61). Conversely, a machine learning marker for dementia in electronic health records failed to improve detection, yielding an adjusted odds ratio of 0.84 (95% CI 0.63-1.11).
Diagnostic accuracy also depended heavily on the clinical use case. Algorithmic skin checks demonstrated a strong AUROC of 0.960 for melanoma discrimination. Yet glaucoma screening tools reached an AUROC of only 0.80 with a modest 65% sensitivity.
Earlier research on AI decision support in primary care documented widespread pilot implementations, but real-world reliability remains uneven across specialties.
Process gains, zero health benefit
Process metrics did show minor observational improvements. Nonrandomized before-after studies reported urinary tract infection treatment success rising from 75% to 80%. A diabetic retinopathy tool changed screening criteria for 3 of 4 general practitioners, and clinicians reported lower estimated record review times during asthma care.
Work on augmenting physician decision-making suggests workflow speed attracts busy clinics. However, faster charting does not automatically yield better healing.
When researchers evaluated actual patient health, the supportive evidence vanished:
- Neither trial assessing patient-important outcomes demonstrated benefit from AI decision support.
- One musculoskeletal trial showed functional outcomes favored usual care over the AI-assisted intervention.
- A trial evaluating HIV pre-exposure prophylaxis guidance was null overall.
- A fall prevention tool improved shared decision-making but yielded inconclusive medication changes.
The practical takeaway
High study heterogeneity and inconsistent outcome measurement prevent causal conclusions about why these tools fail to move the needle. Retrospective accuracy and minor time savings cannot prove software replaces or improves clinical judgment.
Health systems evaluating AI tools must demand trials that track long-term health outcomes alongside diagnostic accuracy. Software that spots disease without improving patient health adds cost and noise without adding value.
Read the full systematic review in the Journal of Medical Internet Research.
