A new clinical trial reveals that giving patients ChatGPT before or after their specialist visits does not reduce their anxiety or help them make better treatment decisions.
Health tech developers assume that putting an AI chatbot in a patient’s hands will instantly demystify complex medical choices. But what if the AI is just adding to the noise?
This randomized trial challenges the industry’s obsession with self-service patient education. For years, the digital health sector has assumed that more information automatically leads to more confident patients. This study suggests that simply handing patients a raw large language model does nothing to ease their clinical anxiety or clarify their treatment paths.
Researchers at a single center evaluated 125 patients presenting with prolapse, incontinence, or lower urinary tract symptoms between July and December 2024. The cohort was mostly White (78.2%) with at least a high school education (92.8%), and almost none (98.8%) had ever used a medical chatbot before. Patients were split into three groups: those using ChatGPT-4o before their visit, those using it after, and a control group receiving usual care.
No change in conflict
Despite high hopes for interactive patient tools, the trial found no statistical difference in how patients made decisions. The researchers measured several key metrics across the groups:
- Decisional conflict scores did not differ immediately after the clinic visit (P = 0.691).
- Long-term decisional conflict remained virtually identical at three months (P = 0.875).
- The trial maintained high engagement, with an attrition rate of just 3.2% at three months.
- Patients in the intervention groups were more likely to continue using chatbots at home by month three (P = 0.011).
The reading level barrier
The disconnect between technology and patient comprehension comes down to how the AI communicates. The chatbot transcripts revealed that the AI’s responses averaged an 11th-to-12th-grade reading level. This is far too complex for standard patient communication, which typically aims for a sixth-grade level to ensure broad accessibility.
This mismatch explains why patient satisfaction and understanding of their diagnosis did not improve. The finding mirrors a related study on patient decision-making and AI chatbots, which also found that raw AI outputs often fail to translate into better clinical comprehension. When AI speaks in academic jargon, it becomes a barrier rather than an aid.
Rethinking the digital front door
This matters because health systems are rapidly integrating AI assistants to triage patients and reduce clinical workloads. If these tools merely dump high-level medical text onto anxious patients, they are not solving the clinical bottleneck. They are simply outsourcing the explanation.
To make these tools work, developers must force AI to simplify its language. Until chatbots can dynamically match a patient’s literacy level, they will remain a novel distraction rather than a clinical tool.
Read the full study in Urogynecology.