Large language models can process qualitative health data six times faster than humans, but they trade away the narrative depth needed to understand complex patient experiences.
Qualitative health research often stalls during the dozens of manual hours required to code and synthesize transcript data. Speeding up this workflow with software offers obvious operational advantages. Yet rushing the analytic process risks flattening the nuanced reality of vulnerable patient groups.
Researchers examined this tradeoff using data from a trial (NCT05437939) evaluating recovery after obstetric fistula. The study included 17 Ugandan women recovering from female genital fistula surgery who completed hour-long semistructured interviews about their physical, mental, and overall health trajectories. Following translation and transcription, human analysts compared their thematic analysis against Versa, a UCSF-developed model powered by ChatGPT-4o, using both inductive and deductive coding strategies.
The automated tool drastically reduced research labor. However, it failed to match the analytical granularity of human investigators.
Speed Gains Versus Lost Nuance
The comparative trial highlighted clear trade-offs between processing speed and contextual accuracy:
- Time efficiency: Human analysis required approximately 15 hours to complete, while Versa produced its analysis in about 2.5 hours.
- Code granularity: During inductive coding, Versa identified 39 codes, whereas human researchers developed a broader set of 54 codes.
- Narrative depth: While Versa accurately captured general interview themes, human synthesis produced greater thematic robustness by incorporating compelling excerpts and detailed contextual range.
This stark disparity in output shows that AI current models cannot fully replace human thematic synthesis. While Versa quickly grouped core concepts, it lacked the interpretive sensitivity necessary to capture the complex lived experiences of surgical recovery.
That disconnect changes how health systems should deploy language models in observational research. AI can handle baseline data organization, but relying on software alone creates a shallow reading of patient experience.
The Practical Takeaway
The operational reality also includes important workflow limitations. Versa still required extensive human labor to write appropriate prompts and prepare inputs before generating outputs. Because the trial analyzed a small cohort of 17 participants, these findings show current tool boundaries rather than a universal baseline for all qualitative clinical software.
Research teams should treat AI tools strictly as draft-stage assistants rather than independent analysts. Using software to execute initial data passes cuts early administrative time, but human oversight remains mandatory to protect interpretive rigor and capture full patient context.
