AI beats standard hospital blood clot risk scores

New data shows machine learning crushes traditional clinical guidelines for predicting hospital-acquired blood clots, questioning our reliance on manual checklists.

Why do we still rely on manual checklists when lives are on the line? For years, hospitals have used the Padua and IMPROVE scores to spot patients at risk of deadly blood clots. Yet these guidelines often perform little better than a coin toss in diverse patient populations.

This disconnect is the real story.

Clinicians tolerate mediocre risk tools because they are easy to calculate at the bedside. But this new study suggests that sticking to these legacy checklists is no longer defensible. It challenges the medical community to replace manual scoring with automated, data-driven algorithms that run silently in the background.

A massive performance gap

Researchers analyzed 399,624 adult admissions from the MIMIC-IV database spanning 2008 to 2022. To isolate new-onset cases, they excluded patients with a history of venous thromboembolism (VTE). Out of this massive cohort, 1,915 admissions (0.479%) resulted in a radiologically confirmed VTE. The team then tested static models using 57 features from the first 24 hours against traditional scores.

The performance gap was massive. On cross-validation, the XGBoost model achieved an AUC of 0.8751 (95% CI 0.8705-0.8805) and logistic regression hit 0.8428. Meanwhile, the widely used Padua score lagged far behind at 0.6330. This represents a significant out-of-fold jump over Padua (delta AUC +0.2403) and IMPROVE (delta AUC +0.2078), both with P-values under 0.0005.

Proven on unseen data

To prove this was not just overfitting, the researchers evaluated the models on a held-out test set of 70,075 admissions containing 325 events. XGBoost scored 0.8873 and logistic regression scored 0.8641, while Padua managed just 0.6188 and IMPROVE hit 0.6521. The advantage held strong in medical patients, where XGBoost scored 0.8904 compared to Padua’s 0.6317.

This is not the first time researchers have tried to automate this process. Previous efforts, such as a hierarchical machine learning model developed in 2021, have targeted multi-department risk assessment. However, this new study stands out because it relies entirely on routine data collected during the first 24 hours of admission.

The limits of complexity

The study also reveals a crucial lesson about model design: more complex is not always better. Adding dynamic landmark updating with 92 time-updated features did improve performance (delta AUC +0.1194, P < 0.0005). However, a highly complex GRU sequence model actually degraded performance, dropping the AUC by -0.0084 to -0.0114 (P >= 0.42).

This suggests that simple, static machine learning models are the sweet spot for clinical deployment. They offer massive accuracy gains without requiring complex, real-time data pipelines. This aligns with ongoing efforts to build an implementable real-time prognostic model that fits into existing hospital workflows.

What we must confront

We must be honest about the limitations. This was a retrospective study on a single database, meaning the models require external validation and local recalibration before clinical use. Sensitivity analyses restricting the data to VTE diagnosed after 24 hours (627 events) and including prior-VTE admissions (2,145 events) still preserved the machine learning advantage (delta AUC +0.1031 and +0.2323, both P < 0.0005).

The clinical implication is clear. Continuing to rely on manual, low-accuracy checklists like Padua is a disservice to patients when automated EHR data can predict clots with far greater accuracy.

Read the full preprint in medRxiv.