⚡ Quick Summary
This study evaluated ChatGPT’s effectiveness in answering public inquiries about the effects of smoking on oral health, revealing that 78.1% of responses were rated as “very useful” or “useful.” However, the model showed limitations in providing actionable guidance, which is crucial for behavioral change.
🔍 Key Details
- 📊 Questions analyzed: 119 smoking-related dental health inquiries
- 🧩 Domains covered: Periodontal conditions, teeth and health, oral hygiene and breath, oral soft tissues, and oral surgery
- ⚙️ AI Model: ChatGPT 3.5
- 🏆 Evaluation metrics: Usefulness, readability, quality, reliability, understandability/actionability
- 📈 Statistical methods: ANOVA, Kruskal-Wallis, Mann-Whitney U, Spearman correlation tests
🔑 Key Takeaways
- 🗣️ ChatGPT provided reliable information on smoking’s impact on oral health.
- 📊 36.1% of responses were rated as “very useful,” and 42.0% as “useful.”
- 🧩 Oral hygiene and breath questions received the highest usefulness scores (p = .002).
- 📉 35.3% of responses were very difficult to read, particularly in surgery-related questions (p < .001).
- 🔍 Overall quality of responses was moderate to good.
- ⚠️ Actionability was limited, with only 23.5% of responses achieving high actionability (> 70%).
- 🌐 Study published in BMC Oral Health, highlighting the role of AI in health education.
📚 Background
The integration of Artificial Intelligence (AI) in healthcare is rapidly evolving, with language models like ChatGPT being utilized to disseminate health information. Smoking is a major risk factor for various oral health issues, including periodontal disease, tooth loss, and oral cancer. Understanding how AI can effectively communicate these risks is crucial for public health education.
🗒️ Study
This study aimed to assess ChatGPT’s ability to respond to public inquiries regarding the effects of smoking on oral health. A total of 119 questions were generated and categorized into five domains. The responses from ChatGPT 3.5 were evaluated using various metrics to determine their usefulness, readability, quality, reliability, and understandability/actionability.
📈 Results
The findings indicated that ChatGPT’s responses were generally rated as “very useful” or “useful” for the majority of questions. However, a significant portion of the responses (35.3%) was found to be very difficult to read, particularly in the context of oral surgery inquiries. While the overall quality and reliability of the information were moderate to good, the limited actionability of the responses poses challenges for individuals seeking personalized health guidance.
🌍 Impact and Implications
The results of this study underscore the potential of AI, particularly ChatGPT, in providing reliable health information. However, the limitations in actionability highlight the need for further development in AI models to offer more personalized and actionable health advice. This could significantly enhance public health education and support individuals in making informed decisions regarding their oral health and smoking cessation.
🔮 Conclusion
This study illustrates the promising role of AI in health education, particularly in addressing smoking’s impact on oral health. While ChatGPT provides valuable information, the lack of actionable guidance limits its effectiveness in promoting behavioral change. Continued research and development in AI technologies are essential to enhance their applicability in healthcare settings, ultimately leading to better health outcomes.
💬 Your comments
What are your thoughts on the use of AI like ChatGPT in health education? Do you believe it can effectively support behavioral change? 💬 Share your insights in the comments below or connect with us on social media:
Assessing ChatGPT’s suitability in responding to the public’s inquires on the effects of smoking on oral health.
Abstract
BACKGROUND: With the growing use of Artificial Intelligence (AI) in healthcare, language models like ChatGPT are increasingly being used to provide health information. Given smoking’s significant impact on oral health, including periodontal disease, tooth loss, oral cancer, and impaired healing-this study assesses ChatGPT’s effectiveness in answering inquiries about the effects of smoking on oral health.
METHODS: A total of 119 smoking-related dental health questions were generated using online tools and categorized into five domains: periodontal conditions, teeth and health, oral hygiene and breath, oral soft tissues, and oral surgery. ChatGPT 3.5 responses were evaluated for usefulness, readability (FKG index and, SMOG index), quality (Global Quality Scale), reliability (CLEAR tool), and understandability/actionability (PEMAT). Statistical analyses included ANOVA, Kruskal-Wallis, Mann-Whitney U, and Spearman correlation tests using SPSS v20.
RESULTS: ChatGPT responses were rated as “very useful” (36.1%) or “useful” (42.0%) for most questions. The oral hygiene and breath category received the highest usefulness scores (p =.002). Readability scores indicated that 35.3% of responses were very difficult to read. Surgery-related questions were significantly harder to read (p <.001). The overall quality and reliability of responses was moderate to good. Although understandability was generally high, actionability was limited, with only 23.5% of responses achieving high actionability (> 70%).
CONCLUSION: ChatGPT provides reliable, high-quality information on smoking’s impact on oral health, suitable for general health education. However, it lacks actionable, personalized guidance, limiting its effectiveness in supporting behavioral change and health management.
Author: [‘Alnsour MM’, ‘Alenezi R’, ‘Barakat M’, ‘Al-Omiri MK’]
Journal: BMC Oral Health
Citation: Alnsour MM, et al. Assessing ChatGPT’s suitability in responding to the public’s inquires on the effects of smoking on oral health. Assessing ChatGPT’s suitability in responding to the public’s inquires on the effects of smoking on oral health. 2025; 25:1207. doi: 10.1186/s12903-025-06377-5