⚡ Quick Summary
This study evaluated the methodological and reporting quality of machine learning (ML) studies in oncology, focusing on adherence to established reporting guidelines. The findings revealed significant deficiencies in reporting quality, particularly in areas such as sample size calculation and data quality documentation.
🔍 Key Details
- 📊 Dataset: 45 studies on cancer diagnosis, treatment, and prognosis
- 🧩 Cancer types studied: Breast cancer (15.6%), Lung cancer (15.6%), Liver cancer (11.1%)
- ⚙️ Guidelines assessed: CREMLS, TRIPOD-AI, PROBAST
- 🏆 Best-performing models: Random Forest and XGBoost (17.8% each)
🔑 Key Takeaways
- 📊 Reporting quality in ML studies for oncology is often inadequate.
- 💡 89% of studies showed a low overall risk of bias according to PROBAST.
- 📝 Deficiencies were noted in sample size calculations and data quality reporting.
- 🤖 Random Forest and XGBoost were the most frequently reported models.
- 🌍 The study highlights the need for improved transparency in ML research.
- 📈 Guidance provided for researchers to enhance reporting quality.
- 🔍 Focus on cancer types like breast, lung, and liver indicates prevalent research areas.
📚 Background
The integration of machine learning in oncology has the potential to transform cancer diagnosis, treatment, and prognosis. However, the rapid evolution of this field necessitates rigorous standards for reporting and methodology to ensure that findings are reliable and applicable in clinical settings. This study aims to address these concerns by evaluating the quality of reporting in recent ML studies.
🗒️ Study
Conducted between February 2024 and January 2025, this study systematically reviewed the most recent ML studies in oncology. A total of 45 primary studies were selected, focusing on their adherence to the Consolidated Reporting Guidelines for Prognostic and Diagnostic Machine Learning Models (CREMLS), TRIPOD-AI, and PROBAST. Two independent reviewers assessed the studies for reporting quality and risk of bias.
📈 Results
The evaluation revealed that while 89% of the studies exhibited a low overall risk of bias, significant deficiencies were identified in reporting quality. Key areas of concern included inadequate sample size calculations, insufficient documentation of data quality, and lack of clarity regarding model performance heterogeneity. The most frequently utilized ML models were Random Forest and XGBoost, each appearing in 17.8% of the studies.
🌍 Impact and Implications
The findings of this study underscore the critical need for improved reporting standards in ML research within oncology. By addressing the identified deficiencies, researchers can enhance the reliability of their studies, ultimately leading to better clinical applications and patient outcomes. The emphasis on transparency and methodological rigor is essential for fostering trust in ML technologies in healthcare.
🔮 Conclusion
This study highlights the importance of methodological quality and transparency in machine learning studies related to cancer. By providing a comprehensive evaluation of reporting practices, it serves as a valuable resource for researchers aiming to improve the quality of their work. The future of ML in oncology holds great promise, and addressing these reporting deficiencies will be crucial for its successful integration into clinical practice.
💬 Your comments
What are your thoughts on the current state of machine learning in oncology? How can we further improve reporting standards in this rapidly evolving field? 💬 Join the conversation in the comments below or connect with us on social media:
Methodological and reporting quality of machine learning studies on cancer diagnosis, treatment, and prognosis.
Abstract
This study aimed to evaluate the quality and transparency of reporting in studies using machine learning (ML) in oncology, focusing on adherence to the Consolidated Reporting Guidelines for Prognostic and Diagnostic Machine Learning Models (CREMLS), TRIPOD-AI (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis), and PROBAST (Prediction Model Risk of Bias Assessment Tool). The literature search included primary studies published between February 1, 2024, and January 31, 2025, that developed or tested ML models for cancer diagnosis, treatment, or prognosis. To reflect the current state of the rapidly evolving landscape of ML applications in oncology, fifteen most recent articles in each category were selected for evaluation. Two independent reviewers screened studies and extracted data on study characteristics, reporting quality (CREMLS and TRIPOD+AI), risk of bias (PROBAST), and ML performance metrics. The most frequently studied cancer types were breast cancer (n=7/45; 15.6%), lung cancer (n=7/45; 15.6%), and liver cancer (n=5/45; 11.1%). The findings indicate several deficiencies in reporting quality, as assessed by CREMLS and TRIPOD+AI. These deficiencies primarily relate to sample size calculation, reporting on data quality, strategies for handling outliers, documentation of ML model predictors, access to training or validation data, and reporting on model performance heterogeneity. The methodological quality assessment using PROBAST revealed that 89% of the included studies exhibited a low overall risk of bias, and all studies have shown a low risk of bias in terms of applicability. Regarding the specific AI models identified as the best-performing, Random Forest (RF) and XGBoost were the most frequently reported, each used in 17.8% of the studies (n = 8). Additionally, our study outlines the specific areas where reporting is deficient, providing researchers with guidance to improve reporting quality in these sections and, consequently, reduce the risk of bias in their studies.
Author: [‘Smiley A’, ‘Villarreal-Zegarra D’, ‘Reategui-Rivera CM’, ‘Escobar-Agreda S’, ‘Finkelstein J’]
Journal: Front Oncol
Citation: Smiley A, et al. Methodological and reporting quality of machine learning studies on cancer diagnosis, treatment, and prognosis. Methodological and reporting quality of machine learning studies on cancer diagnosis, treatment, and prognosis. 2025; 15:1555247. doi: 10.3389/fonc.2025.1555247