โก Quick Summary
This study evaluated the effectiveness of large language models (LLMs), specifically ChatGPT-4o, in providing accurate information on cardiovascular primary and secondary prevention. While the model achieved a 75% accuracy rate, the readability of the content was found to be difficult for users.
๐ Key Details
- ๐ Study Focus: Evaluation of LLM-generated information on cardiovascular prevention.
- ๐ง Technology Used: ChatGPT-4o.
- ๐ฉบ Expert Evaluation: Responses assessed by cardiologists using a Likert scale.
- ๐ Readability Assessment: Flesch Reading Ease Score (FRES).
๐ Key Takeaways
- ๐ก ChatGPT-4o provided accurate responses for 75% of the items related to myocardial infarction risk prevention.
- ๐ Readability scores indicated that the content was difficult to understand.
- ๐ฉบ Specialized clinical topics showed lower accuracy and readability compared to general topics.
- ๐ Clinical oversight is essential to ensure optimal patient outcomes.
- ๐ The study highlights the role of AI in health education alongside traditional healthcare professionals.
๐ Background
Cardiovascular diseases remain the leading cause of mortality worldwide, with ischaemic heart disease being a significant contributor. Effective primary and secondary prevention strategies are crucial for reducing the incidence and recurrence of conditions like acute myocardial infarction. As healthcare evolves, the Internet and AI tools are becoming increasingly important in disseminating health education.
๐๏ธ Study
This observational study aimed to assess the accuracy and readability of information generated by ChatGPT-4o regarding cardiovascular prevention strategies. The researchers posed tailored questions about the risk of acute myocardial infarction and evaluated the model’s responses through expert cardiologists and readability metrics.
๐ Results
The findings revealed that ChatGPT-4o provided comprehensive and accurate responses for 15 out of 20 items, translating to a 75% accuracy rate. However, the readability scores were low, indicating that the content was challenging for users to comprehend. Specialized clinical topics were particularly noted for their lower accuracy and readability.
๐ Impact and Implications
The implications of this study are significant, as it underscores the potential of AI tools like ChatGPT in enhancing health education. However, the challenges related to readability highlight the need for clinical oversight to ensure that patients receive understandable and actionable information. This balance is essential for improving patient outcomes in cardiovascular health.
๐ฎ Conclusion
This study illustrates the promising role of large language models in providing accurate information on cardiovascular prevention. While ChatGPT-4o demonstrated a high level of accuracy, the difficulty in readability suggests that further refinement is necessary. Continued research and development in this area could lead to more effective health education tools that empower patients and healthcare providers alike.
๐ฌ Your comments
What are your thoughts on the use of AI in health education? Do you believe that tools like ChatGPT can effectively support healthcare professionals? Let’s discuss! ๐ฌ Leave your comments below or connect with us on social media:
Large language models to develop evidence-based strategies for primary and secondary cardiovascular prevention.
Abstract
AIMS: Cardiovascular diseases are the leading global cause of mortality, with ischaemic heart disease contributing significantly to the burden. Primary and secondary prevention strategies are essential to reducing the incidence and recurrence of acute myocardial infarction. Healthcare professionals are no longer the sole source of health education; the Internet, including tools powered by artificial intelligence, is also widely utilized. This study evaluates the accuracy and the readability of large language model (LLM)-generated information on cardiovascular primary and secondary prevention.
METHODS AND RESULTS: An observational study assessed LLM’s responses to two tailored questions about acute myocardial infarction risk prevention. The LLM used was ChatGPT (4o version). Expert cardiologists evaluated the accuracy of each response using a Likert scale, while readability was assessed with the Flesch Reading Ease Score (FRES). ChatGPT-4o provided comprehensive and accurate responses for 15 out of 20 (75%) of the items. Readability scores were low, with median FRES indicating that both primary and secondary prevention content were difficult to understand. Specialized clinical topics exhibited lower accuracy and readability compared to the other topics.
CONCLUSION: The current study demonstrated that ChatGPT-4o provided accurate information on primary and secondary prevention, although its readability was assessed as difficult. However, clinical oversight still remains critical to bridge gaps in accuracy and readability and ensure optimal patient outcomes.
Author: [‘Lorenzoni G’, ‘Zanotto C’, ‘Sordo A’, ‘Cipriani A’, ‘Perazzolo Marra M’, ‘Tona F’, ‘Gasparini D’, ‘Gregori D’]
Journal: Eur Heart J Digit Health
Citation: Lorenzoni G, et al. Large language models to develop evidence-based strategies for primary and secondary cardiovascular prevention. Large language models to develop evidence-based strategies for primary and secondary cardiovascular prevention. 2025; 6:1069-1075. doi: 10.1093/ehjdh/ztaf085