โก Quick Summary
This review highlights the limitations of artificial intelligence (AI) and machine learning in rheumatology, emphasizing the gap between high expectations and actual performance in clinical settings. It underscores the need for realistic expectations and robust methodologies to improve AI applications in healthcare.
๐ Key Details
- ๐ Focus: The impact of AI and machine learning in rheumatology
- ๐งฉ Key Issues: Overestimation of effects in early studies, methodological flaws
- โ๏ธ Examples: Historical parallels with antioxidant supplementation and the Human Genome Project
- ๐ Findings: AI models often suffer from small datasets and poor validation
๐ Key Takeaways
- ๐ Early studies tend to overestimate the effectiveness of interventions.
- ๐ The ‘cursed auction’ analogy explains systematic overestimations in small studies.
- ๐งฌ Individualized clinical risk is constrained by the reference class problem.
- ๐ Many AI models in rheumatology are hindered by methodological issues.
- ๐ซ COVID-19 prediction models demonstrated the collapse of retrospective performance in real-world scenarios.
- ๐ AI can provide insights but struggles with noisy clinical data.
- ๐ Future progress requires large datasets and rigorous validation.
- ๐ ๏ธ Focus should be on interpretable tools for well-defined patient subgroups.
๐ Background
The integration of artificial intelligence and machine learning into medical practice has been heralded as a transformative advancement. However, the reality in fields like rheumatology shows a significant gap between the anticipated benefits and the actual impact on daily clinical practice. Historical examples, such as the Human Genome Project and studies on vitamin D, reveal a pattern where initial promising results often diminish in larger, more rigorous studies.
๐๏ธ Study
This review critically examines the current state of AI in rheumatology, drawing on historical parallels and recent methodological evidence. It highlights the challenges faced by AI models, including issues related to small and heterogeneous datasets, overfitting, and inadequate handling of missing data. The authors argue for a more cautious approach to AI predictions in clinical settings.
๐ Results
The findings indicate that many AI models in rheumatology are not yet ready for widespread clinical application. The failure of COVID-19 prediction models and the neutral trial of the Ada diagnostic assistant serve as cautionary tales, illustrating how strong performance in retrospective studies can fail in real-world scenarios. The review emphasizes the need for large representative datasets and rigorous validation to ensure the reliability of AI applications.
๐ Impact and Implications
The implications of this review are significant for the future of AI in healthcare. While AI has the potential to generate valuable insights and support specific tasks, it cannot yet overcome the inherent limitations of clinical data. The authors advocate for a shift in focus towards developing robust, interpretable tools that enhance decision-making for populations and well-defined patient subgroups, rather than striving for precise individual predictions.
๐ฎ Conclusion
This review serves as a reminder of the critical need for skepticism regarding the promises of AI in medicine. As we continue to explore the potential of these technologies, it is essential to maintain realistic expectations and prioritize rigorous methodologies. The future of AI in rheumatology and beyond will depend on our ability to navigate these challenges effectively.
๐ฌ Your comments
What are your thoughts on the current state of AI in healthcare? Do you believe we are setting realistic expectations? Let’s engage in a discussion! ๐ฌ Share your insights in the comments below or connect with us on social media:
Why we need to maintain a critical view on big data and artificial intelligence predictions.
Abstract
Artificial intelligence (AI) and machine learning are widely promoted as transformative tools for medical practice, yet their impact in daily rheumatology remains limited. This review examines the gap between expectations and reality using historical parallels, conceptual considerations, and recent methodological evidence. Experiences with antioxidant supplementation, vitamin D, the microbiome, and the Human Genome Project illustrate a recurring pattern: early studies report large effects that diminish or disappear in larger, higher-quality studies. Meta-epidemiological work and the ‘cursed auction’ analogy explain why early and small studies systematically overestimate effects. Conceptually, individualized clinical risk remains a group-based construct, constrained by the reference class problem and irreducible uncertainty. Methodologically, many AI models in rheumatology suffer from small and heterogeneous datasets, overfitting, inadequate handling of missing data, poor calibration, and limited external or prospective validation. The failure of COVID-19 prediction models and the neutral trial of the Ada diagnostic assistant in rheumatology illustrate how strong retrospective performance often collapses in real-world use. In contrast, AI performs well in high signal-to-noise domains with abundant, structured data. Overall, AI can generate valuable insights and support narrowly defined tasks, but it cannot yet overcome the fundamental limits of noisy clinical data and group-based risk. Progress in rheumatology will require realistic expectations, large representative datasets, transparent methods, rigorous validation, and a focus on robust, interpretable tools that improve decisions for populations and well-defined patient subgroups rather than precise individual prediction.
Author: [‘Temiz A’, ‘Tascilar K’]
Journal: Curr Opin Immunol
Citation: Temiz A and Tascilar K. Why we need to maintain a critical view on big data and artificial intelligence predictions. Why we need to maintain a critical view on big data and artificial intelligence predictions. 2026; 100:102776. doi: 10.1016/j.coi.2026.102776