Adding text embeddings from clinical notes to standard lab data helps algorithms separate tricky liver conditions that look alike on blood tests.
Standard machine learning models often struggle to differentiate autoimmune liver disease, drug-induced liver injury, and chronic hepatitis B. Lab tests for these conditions frequently overlap, leaving critical diagnostic clues buried in unstructured clinical narratives. Combining text embeddings from large language models with tabular lab data directly addresses this gap.
A retrospective study at Beijing Youan Hospital analyzed electronic health records from 7,543 patients treated between 2010 and 2025. Researchers evaluated three language models—Qwen3, Huatuo-o1, and II-Medical—to extract semantic features from unstructured examination reports and observations. This text processing strategy aligns with established research in EHR-driven phenotyping, which emphasizes extracting diagnostic signals from clinical text.
Clear accuracy gains
Integrating text embeddings with standard machine learning outperformed traditional models reliant solely on structured blood work. The integrated approach maintained its edge during temporal validation, where models trained on 2010–2019 data were evaluated on cases from 2020–2025.
The comparative metrics highlight consistent gains across diagnostic tasks:
- 3-class etiology task: Integrated models achieved macro F1-scores of 0.835 to 0.837 and accuracies of 0.925 to 0.929, compared to 0.791 F1 and 0.893 accuracy for lab-only baselines.
- 4-class subtype task: For fine-grained autoimmune classification, integrated models reached F1-scores of 0.717 to 0.734 and accuracies of 0.920 to 0.922, beating the baseline F1 of 0.665 and accuracy of 0.874.
- Zero-shot reasoning: Direct zero-shot LLM prompts and lightweight natural language processing encoders performed below the embedding-based hybrid models.
That performance gap matters for prospective clinical tool design.
Rather than relying on language models to make autonomous diagnostic decisions, the trial demonstrates that their strongest role is structured feature extraction. Converting narrative notes into dense mathematical vectors gives classical classifiers the contextual signals missing from standard blood panels.
Real-world limitations
These findings come with an important practical caveat. The study strictly excluded patients with overlapping disease etiologies, selecting only clear-cut cases with single diagnoses from a single specialty hospital. In routine clinical practice, patients frequently present with complex, co-occurring pathologies that confound algorithmic predictions.
Because artificial cohort filtering inflates performance, these accuracy numbers will likely drop in heterogeneous, real-world populations. Healthcare systems should view this framework as a method for data enrichment rather than proof of automated diagnostic readiness.
Read the full findings in the Journal of Medical Internet Research.
