⚡ Quick Summary
This study investigates the use of machine learning algorithms to automate the extraction of progression-free survival (PFS) metrics from electronic health records (EHR) in patients with glioblastoma (GBM). The findings reveal that while machine learning methods can derive PFS outcomes, they often do not align with traditional clinical guidelines, highlighting the need for improved data integration and validation.
🔍 Key Details
- 📊 Dataset: EHR data from 92 pathology-proven GBM patients
- 🧩 Features used: 233 corticosteroid prescriptions, 2080 radiology reports, 743 brain MRI scans
- ⚙️ Methods: Frequency analysis, natural language processing (NLP), computer vision (CV)
- 🏆 Comparison: Results compared to manually annotated clinical guideline PFS metrics
🔑 Key Takeaways
- 📊 Machine learning can extract tumor progression outcomes from existing EHR data.
- 💡 Standalone progression rates were 63% (prescription), 78% (NLP), and 54% (CV).
- 🏆 Manual clinical guidelines reported a 99% progression rate using integrated data sources.
- ⏳ Timing differences: Prescription method identified progression 5.2 months later, while NLP and CV identified it earlier by 2.6 and 6.9 months, respectively.
- 🧠 Clinical intuition: Automation may not align with human intuition regarding lesion growth indicators.
- 🔄 Need for integration: Improved data source integration and validation is essential for accurate PFS metrics.
- 📈 Resource burdens: ML methods face challenges related to availability bias and pre-processing requirements.
📚 Background
Progression-free survival (PFS) is a vital endpoint in cancer management, providing insights into treatment efficacy. However, the traditional methods for calculating PFS are often subjective and labor-intensive, leading to missing data in publicly available datasets. The advent of machine learning offers a promising avenue to automate this process, potentially enhancing the accuracy and efficiency of PFS metric extraction from electronic health records.
🗒️ Study
The study analyzed EHR data from 92 patients diagnosed with glioblastoma (GBM). Researchers developed three distinct methods to derive clinical PFS: frequency analysis of corticosteroid prescriptions, natural language processing (NLP) of radiology reports, and computer vision (CV) volumetric analysis of brain MRI scans. The outputs from these methods were then compared to manually annotated clinical guidelines to assess their effectiveness.
📈 Results
The results indicated that the machine learning methods produced varying standalone progression rates: 63% for the prescription method, 78% for NLP, and 54% for CV. In contrast, the manual clinical guidelines achieved a remarkable 99% progression rate. Notably, the prescription method identified tumor progression an average of 5.2 months later than the clinical standard, while NLP and CV methods detected progression earlier by 2.6 and 6.9 months, respectively.
🌍 Impact and Implications
The findings from this study underscore the potential of machine learning to revolutionize how we derive clinical outcomes from EHR data. However, the discrepancies between automated methods and traditional clinical guidelines highlight the importance of integrating diverse data sources and validating machine learning outputs. As we move forward, it is crucial to revisit clinical criteria in conjunction with the development of multi-modal machine learning algorithms to ensure alignment with clinical intuition and practice.
🔮 Conclusion
This study illustrates the transformative potential of machine learning in automating the extraction of PFS metrics from electronic health records. While promising, the results also reveal significant challenges that must be addressed, including data integration and validation. As we continue to explore the intersection of technology and healthcare, ongoing research is essential to refine these methods and improve patient outcomes in cancer management.
💬 Your comments
What are your thoughts on the use of machine learning for extracting clinical outcomes from electronic health records? We would love to hear your insights! 💬 Leave your comments below or connect with us on social media:
From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities.
Abstract
BACKGROUND: Progression free survival (PFS) is a critical clinical outcome endpoint during cancer management and treatment evaluation. Yet, PFS is often missing from publicly available datasets due to the current subjective, expert, and time-intensive nature of generating PFS metrics. Given emerging research in multi-modal machine learning (ML), we explored the benefits and challenges associated with mining different electronic health record (EHR) data modalities and automating extraction of PFS metrics via ML algorithms.
METHODS: We analyzed EHR data from 92 pathology-proven GBM patients, obtaining 233 corticosteroid prescriptions, 2080 radiology reports, and 743 brain MRI scans. Three methods were developed to derive clinical PFS: 1) frequency analysis of corticosteroid prescriptions, 2) natural language processing (NLP) of reports, and 3) computer vision (CV) volumetric analysis of imaging. Outputs from these methods were compared to manually annotated clinical guideline PFS metrics.
RESULTS: Employing data-driven methods, standalone progression rates were 63% (prescription), 78% (NLP), and 54% (CV), compared to the 99% progression rate from manually applied clinical guidelines using integrated data sources. The prescription method identified progression an average of 5.2 months later than the clinical standard, while the CV and NLP algorithms identified progression earlier by 2.6 and 6.9 months, respectively. While lesion growth is a clinical guideline progression indicator, only half of patients exhibited increasing contrast-enhancing tumor volumes during scan-based CV analysis.
CONCLUSION: Our results indicate that data-driven algorithms can extract tumor progression outcomes from existing EHR data. However, ML methods are subject to varying availability bias, supporting contextual information, and pre-processing resource burdens that influence the extracted PFS endpoint distributions. Our scan-based CV results also suggest that the automation of clinical criteria may not align with human intuition. Our findings indicate a need for improved data source integration, validation, and revisiting of clinical criteria in parallel to multi-modal ML algorithm development.
Author: [‘Chappidi S’, ‘Belue MJ’, ‘Harmon SA’, ‘Jagasia S’, ‘Zhuge Y’, ‘Tasci E’, ‘Turkbey B’, ‘Singh J’, ‘Camphausen K’, ‘Krauze AV’]
Journal: PLOS Digit Health
Citation: Chappidi S, et al. From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities. From manual clinical criteria to machine learning algorithms: Comparing outcome endpoints derived from diverse electronic health record data modalities. 2025; 4:e0000755. doi: 10.1371/journal.pdig.0000755