Title: AI calculates lupus severity from clinical notes

Automating lupus tracking with open-source language models could finally bring standardized disease monitoring to busy clinics.

How do you track a disease that attacks different organs every week without burning out the doctors treating it? Systemic lupus erythematosus (SLE) requires clinicians to manually calculate a complex severity score called the SLEDAI-2K. It is a tedious process that often gets skipped in busy, real-world clinics.

This is where the real bottleneck in autoimmune care lies. If we cannot track disease activity consistently, we cannot predict organ damage before it happens. This trial suggests that we might not need human specialists to do the manual math anymore.

Researchers built a framework using open-source large language models to extract these scores directly from electronic health records. They trained the system on a specialist-verified dataset of 658 clinical notes and tested it externally on 56 discharge summaries from the MIMIC-IV database. The top-performing setup used a two-layered GPT-OSS-120B model paired with a verifier.

How the model performed

• Achieved a 94.2% micro-F1 score for classifying disease symptoms on internal clinical notes.
• Reached an 86% exact match rate for overall SLEDAI-2K scores on the internal dataset.
• Dropped to an 87.7% micro-F1 and a 64% exact match rate during external validation on unseen hospital summaries.
• Analyzed 2,576 serial notes from 108 patients tracked over 18.3 years to prove its long-term clinical utility.

The drop in external validation performance from 86% to 64% exact match is the critical detail here. It exposes the classic brittleness of clinical AI when moved to a new hospital system with different writing styles. Yet, the real value of this tool is not perfect daily scoring. It is long-term trend spotting.

When the AI reviewed nearly two decades of patient history, its classifications actually predicted major medical crises. Patients that the AI flagged as having sustained low disease activity had significantly fewer complications. Specifically, they showed lower rates of stage 3 chronic kidney disease (p = 0.0053), required less kidney replacement therapy (p = 0.044), and had fewer hospitalizations (p = 0.021).

The reality of clinical deployment

This proves that an AI does not need to be flawless in its daily math to be clinically useful over a patient’s lifetime. By identifying long-term trends, the model acts as an early warning system for kidney failure. It shifts the AI conversation from perfect diagnostic accuracy to practical risk prediction.

However, relying on open-source models locally is a double-edged sword. It keeps patient data private, which is vital for hospital compliance. But running a 120-billion parameter model requires serious local computing power that small clinics simply do not have. For now, this technology remains a powerful research tool rather than a plug-and-play clinic assistant.

Read the full preprint on medRxiv.