⚡ Quick Summary

This study evaluated the performance of five large language models (LLMs) in providing orthodontic consultation for patients with periodontitis. Notably, Grok-3 excelled in reliability and quality, while DeepSeek-V3 was the most readable, highlighting the potential of AI in patient education.

🔍 Key Details

• 📊 Models evaluated: ChatGPT-4o, DeepSeek-V3, Claude-Sonnet-4, Gemini-2.0 Flash, Grok-3
• 🔍 Evaluation metrics: Reliability (mDISCERN), Quality (GQS), Readability (FRE, FKGL)
• 📅 Study duration: Responses to 30 frequently asked questions sourced from social media and health websites
• 📈 Statistical analysis: Linear mixed-effects models with Bonferroni’s adjustment

🔑 Key Takeaways

• 📊 Grok-3 achieved the highest reliability (mDISCERN: 4.20) and quality (GQS: 4.38).
• 📉 Claude-Sonnet-4 scored the lowest in both reliability (mDISCERN: 3.54) and quality (GQS: 3.63).
• 📖 DeepSeek-V3 was rated as the most readable (FRE: 33.61; FKGL: 10.10).
• 📉 Claude-Sonnet-4 had the least readability (FRE: 4.73; FKGL: 13.72).
• 📚 All models produced responses with university-level readability.
• ⚠️ Caution advised: These models should be considered supplementary resources due to potential misinformation risks.
• 🌍 Study published in: BMC Oral Health.
• 🆔 PMID: 42062980.

📚 Background

The integration of large language models (LLMs) into healthcare has opened new avenues for patient education, particularly in specialized fields like orthodontics. Patients with periodontitis often seek guidance on orthodontic treatment, making it crucial to evaluate the reliability and quality of AI-generated responses to ensure they receive accurate and comprehensible information.

🗒️ Study

This study aimed to assess the performance of five publicly accessible LLM-based chatbots in addressing common inquiries from patients with periodontitis. By compiling thirty frequently asked questions from social media and health-related websites, researchers evaluated each model’s responses for reliability, quality, and readability using established metrics.

📈 Results

The findings revealed significant performance differences among the evaluated models (P < 0.001). Grok-3 emerged as the top performer in both reliability and quality, while DeepSeek-V3 was noted for its superior readability. All models, however, maintained a university-level readability standard, indicating their potential utility in patient education.

🌍 Impact and Implications

The results of this study underscore the potential of LLMs in enhancing patient education, particularly for complex topics like orthodontic treatment for periodontitis. While these models can provide valuable information, it is essential to approach their use with caution, recognizing their limitations and the risks of misinformation. As AI continues to evolve, integrating these technologies into patient care could significantly improve the quality of information available to patients.

🔮 Conclusion

This study highlights the promising role of large language models in orthodontic consultations for patients with periodontitis. With Grok-3 demonstrating higher reliability and quality, and DeepSeek-V3 offering more readable responses, these tools can serve as valuable supplementary resources. However, ongoing research and careful implementation are necessary to ensure their safe and effective use in healthcare settings.

💬 Your comments

What are your thoughts on the use of AI in patient education? Do you believe these models can enhance the quality of information provided to patients? 💬 Share your insights in the comments below or connect with us on social media:

• 🐦 X
• 📘 Facebook
• 🔗 LinkedIn