An agentic AI system reviewed hospital readmissions as accurately as human doctors for a fraction of the cost, but the real revelation is how much clinicians disagree with each other.
How much is a doctor’s 15-minute chart review worth if another doctor disagrees with their conclusion? Hospital quality departments spend fortunes manually auditing 30-day readmissions to find preventable errors. This medRxiv preprint shows that an AI agent can do this work for pennies, matching human quality. But it also exposes a deeper truth: “preventability” is not a hard science, but a subjective judgment call.
For years, clinical audits have been bottlenecked by human hours. This trial suggests we can automate the tedious paperwork of chart reviews. However, because human doctors rarely agree on why a patient returned to the hospital, we must rethink what we are actually measuring when we grade hospital quality.
The cost of human consensus
In the primary evaluation, researchers pitted an AI agent against human physicians using a sample of 20 randomly selected adult readmissions from 2025. The AI queried a database of clinical notes, laboratory results, and procedures to complete a structured review rubric. The AI classified 9/20 (45%) of the readmissions as preventable, compared to 19/40 (47.5%) of the physician reviews.
When blinded evaluators graded the reviews on a 1-to-5 scale, the AI actually scored slightly higher than the humans. The AI earned an overall quality rating of 4.35 compared to the physicians’ 4.20, a mean difference of 0.15 (95% CI -0.20 to 0.48; p=0.49). Factuality and usefulness ratings were also highly comparable, and the audit found zero AI hallucinations.
The financial contrast is stark. A physician took a median of 15 minutes to review a single chart, costing an estimated $42.43 in labor. The AI completed the same task for just $0.23 per chart.
The subjectivity problem
The real shock was not the AI’s speed, but the lack of consensus. Agreement on whether a readmission was preventable was low across the board. This was true for AI-to-human comparisons, but equally true for human-to-human comparisons.
This highlights a major challenge in clinical AI design. As explored in a study on cognitive alignment in cardiovascular AI, we must design systems that align with clinical reasoning rather than just chasing a single “correct” answer that humans themselves cannot agree on.
Scaling up the audit
Free from human time constraints, the researchers deployed the AI to audit an expanded cohort of 100 recent readmissions. The system successfully flagged recurring systemic vulnerabilities.
- Post-discharge follow-up: Incomplete or delayed outpatient appointments.
- Inpatient workups: Unresolved clinical questions at the time of discharge.
- Medication safety: Poor transitions and reconciliation errors.
- Indwelling devices: Inadequate monitoring of catheters and lines.
The honest limits
We must look at the limitations. This was a small, single-center study at one academic health system. The AI queried a structured database, not raw, messy legacy systems. Because preventability judgments remain highly variable, this technology cannot run on autopilot. It is a powerful tool to draft reviews and highlight trends, but human oversight remains mandatory.
Read the full preprint on medRxiv.