A new machine learning model can pinpoint which ankle fracture patients are highly likely to develop surgical infections, but its tendency to miss the majority of at-risk cases makes it a dangerous tool if used as a standalone safety net.
How much risk are you willing to tolerate when a missed bone infection could mean amputation? For surgeons treating closed pilon fractures—severe ankle injuries often caused by high-impact trauma—surgical site infections (SSIs) are a constant threat. Identifying which patients will develop these complications before they happen remains a massive clinical challenge.
A new study in Frontiers in Surgery attempts to solve this with machine learning. But the results expose a critical tension in clinical AI. A model that is exceptionally good at avoiding false alarms can still be dangerously blind to the very patients who need help most.
Testing the algorithms
Researchers built and tested their models using data from two distinct hospital groups. The internal dataset included 1,876 patients treated between January 2020 and December 2024, which was split 7:3 for training and testing. To see if the tool could work in the real world, they also tested it on an external cohort of 359 patients treated between August 2024 and September 2025. Out of these groups, the actual infection rates were low but significant, with 74 SSI events (3.9%) in the internal group and 11 SSI events (3.1%) in the external group.
The team used a statistical method called LASSO regression to narrow down a massive list of demographic, laboratory, and inflammatory variables to just 16 key predictors. They tested several algorithms, including logistic regression, decision trees, random forests, XGBoost, and naïve Bayes.
High accuracy, low safety
The random forest (RF) model emerged as the top performer, but its metrics reveal a stark trade-off.
- The RF model achieved a high area under the receiver operating characteristic curve (ROC-AUC) of 0.899 on the internal test set and 0.902 on the external cohort.
- Specificity was remarkably high, reaching 0.987 internally and 0.974 externally, meaning it rarely flagged a healthy patient as high-risk.
- Internal sensitivity was a dismal 0.294, though it rose to 0.636 in the external validation cohort.
- The model’s Brier scores were 0.026 internally and 0.022 externally, with calibration slopes of 1.915 and 1.668.
- A simplified, preoperative-only version of the model still managed ROC-AUCs of 0.884 internally and 0.905 externally.
That sensitivity gap is the real story.
An internal sensitivity of 0.294 means the algorithm missed more than 70% of the patients who actually went on to develop an infection. While the external cohort saw sensitivity rise to 0.636, that still leaves more than a third of infected patients undetected. If doctors rely on this tool to decide who gets extra wound care, the high specificity protects hospital resources, but the low sensitivity abandons vulnerable patients.
The calibration metrics also signal caution. The observed-to-expected ratios of 0.736 internally and 0.762 externally show that the model consistently overestimates risk. Combined with calibration slopes well above 1.0, the tool is mathematically unstable and requires heavy tuning before any real-world deployment.
The clinical reality
In orthopedic trauma, missing an infection is far worse than investigating a false alarm. This study confirms a growing worry in medical AI. Models optimized for overall mathematical accuracy often fail the basic safety requirements of bedside medicine.
For now, this algorithm cannot be used to rule out infection risk. It functions only as an early warning system for a select few. Until researchers can recalibrate the model to catch more infections without completely sacrificing specificity, surgeons must trust their clinical instincts over the software.
Read the full study in Frontiers in Surgery.