A co-designed smartphone app helps people with expressive aphasia turn broken speech into clear sentences.
Patients eager, clinicians urge caution
Large language models can interpret fractured speech, but clinical adoption hinges on balancing processing speed against safety. In early testing with four stroke survivors, the R-SPEAK smartphone app earned a System Usability Scale score of 75 out of 100. Yet while patients expressed high enthusiasm, a focus group of 6 healthcare professionals voiced caution about clinical reliability.
That divide highlights a persistent challenge in digital health tools.
Patients prioritize speed and independence in daily interactions. Clinicians worry about false interpretations and physical access barriers. Post-stroke motor deficits or severe language loss may prevent patients from using screen interfaces effectively without guided training.
Setting limits for speech AI
Researchers built R-SPEAK using a three-phase co-design strategy alongside stroke survivors and medical specialists. Phase 1 established core system features through patient and public involvement sessions using MoSCoW prioritization rankings. In Phase 2, the team evaluated 8 open-weight models on disordered speech transcripts, choosing Mixtral 8x7B to power the initial prototype.
Because large models run slowly on mobile devices, Phase 3 evaluated 12 lightweight models between 0.5 and 4 billion parameters using an LLM-as-a-judge framework. Key benchmark results include:
- Qwen2.5:3B emerged as the top performer, achieving sub-second latency while maintaining high answer relevancy and faithfulness.
- Models with fewer than 1.5 billion parameters suffered from frequent hallucinations, establishing a strict lower limit for safe speech translation.
- Qualitative testing across 5 participants (4 patients, 1 carer) confirmed high perceived usefulness for daily independence.
Small cohorts hide long-term risks
Establishing a 1.5-billion-parameter threshold gives software developers a concrete technical baseline. Mobile deployments often force developers to compress software, but aggressive shrinking introduces dangerous fabrications. For expressive aphasia, an incorrect translation can misrepresent patient needs or medical symptoms.
This evaluation remains strictly exploratory. The prototype was tested in qualitative interviews with just 4 patients, 1 carer, and 6 healthcare professionals. Retrospective benchmarks on public transcripts do not guarantee that AI speech tools improve real-world patient health outcomes or lower clinician workload.
Larger prospective trials must now confirm whether sub-second processing speeds translate into actual functional independence in community settings.
Read the complete research in the Journal of Medical Internet Research.
