A new multimodal AI system automatically drafts ophthalmic ultrasound reports and outperforms leading commercial models in clinical question-answering.
Can an algorithm reliably read the sound waves bouncing off the back of an eyeball? Ophthalmic B-scan ultrasound is vital for diagnosing blinding conditions, but interpreting these grainy images requires years of specialized training. If a clinic lacks an expert on hand, patients wait and diagnoses stall.
This is not just about automation. It is about shifting the bottleneck from human eyes to machine-guided workflows. By pairing image-to-text models with conversational large language models, researchers are attempting to replace the traditional static report with an interactive clinical dialogue.
High marks for accuracy
To build this system, named OphthUS-GPT, researchers trained a two-stage model on a massive retrospective dataset. They analyzed 103,237 ultrasound images and 51,618 diagnostic reports from 51,618 patients at the Affiliated Eye Hospital of Nanchang University. The system uses a fine-tuned BLIP model to first draft clinical findings and then synthesize them into final impressions.
The automated reports matched human clinical reasoning with surprising precision. In text similarity testing, the system achieved a BLEU-1 score of 0.5739, a ROUGE-L of 0.6131, and a CIDEr score of 0.9818. More than 90% of the generated reports received clinical accuracy and completeness scores of 3 or higher from expert evaluators.
For specific diseases, the diagnostic performance was highly reliable:
- Classification accuracy exceeded 90% for vitreous opacities, retinal detachment, posterior scleral staphyloma, and cataracts.
- Overall diagnostic accuracy exceeded 80% across all evaluated disease categories.
- The integrated DeepSeek model achieved statistically superior accuracy compared to Claude, and higher user satisfaction than both Claude and GPT-4 Turbo.
The interactive shift
The real shift here is the conversational layer. Traditionally, an ultrasound report is a static piece of paper. By deploying a local DeepSeek model, clinicians can ask follow-up questions about the images. This interactive setup provides interpretable clinical decision support on demand.
The fact that this open-source local model outperformed proprietary giants like GPT-4 Turbo in user satisfaction suggests a clear path forward. Highly specialized, locally tuned models may be far more practical for clinical environments than massive, generalized cloud APIs.
Real-world limitations
However, the system is not without flaws. While accuracy for common conditions topped 90%, the overall accuracy across all disease categories dropped to just over 80%. This gap indicates that rare or complex ocular pathologies still present a challenge for the model.
Furthermore, the dataset originates from a single institution. This means the system’s generalizability across different ultrasound hardware and diverse patient populations remains unproven. Until multi-center trials validate these results, human experts must remain the final authority.
Read the full study in the Journal Of Big Data.
