Patients who read AI-generated summaries of their brain scans feel significantly more confident, but they do not actually understand their medical results any better.

If a patient feels like they understand a brain scan, does it matter if they actually do? A new clinical trial reveals a dangerous disconnect in patient-facing AI. When we give patients AI-translated medical reports, we might just be feeding them a false sense of security.

This illusion of understanding challenges the rush to integrate large language models into patient portals. While some researchers argue these tools drive patient empowerment, as discussed in a 2026 comparative analysis on medical data interpretation, this new data suggests we are confusing customer satisfaction with actual medical literacy.

Testing the AI summaries

Researchers tested this dynamic using a randomized controlled trial of 2,727 adult participants from the ComPaRe e-cohort. The cohort had a mean age of 52 years and was 75.2% women. Participants were asked to interpret six standardized brain MRI reports for headaches.

The control group of 1,401 people read the raw reports. The intervention group of 1,326 people received the same reports appended with a lay summary generated by the Mistral Small 3.2 model.

The confidence gap

The results show a stark divide between how smart the AI made patients feel and how much they actually grasped.

• Objective comprehension was statistically identical, coming in at 59.4% for the AI group versus 58.3% for the control group, with an odds ratio of 0.97.
• Subjective comprehension doubled, with 50.3% of the AI group believing they understood the report compared to only 24.0% of the control group, with an odds ratio of 3.17.
• Overall satisfaction nearly doubled to 64.9% in the AI group, up from 36.7% in the control group, yielding an odds ratio of 3.26.
• High anxiety saw only a tiny, modest reduction, dropping to 25.1% with AI summaries compared to 26.6% without them, with an odds ratio of 0.92.

The most troubling finding lies in how the AI altered comprehension based on the scan’s actual results. When a scan showed a clear cause for the headache, the AI summary helped, raising correct interpretation from 37.4% to 42.4%.

However, when the MRI was completely normal, the AI actually confused patients. Correct interpretation of normal scans dropped from 76.6% in the control group to 72.5% in the AI group.

This is the real danger. AI summaries can introduce noise into normal results, making healthy patients worry or seek unnecessary follow-ups.

Limits of the study

We must note the study’s limitations. This trial relied on hypothetical headache scenarios within an online cohort, not real-world patients waiting for their own diagnoses. It is also a preprint that has not yet completed peer review.

Even so, the takeaway is clear. Giving patients simplified AI text makes them happy, but it does not make them informed. Until AI tools can explain normal scans without muddying the waters, placing them directly in patient portals is a liability.

Read the full study in medRxiv.