Follow us
🧑🏼‍💻 Research - October 28, 2024

Appropriateness and readability of Google Bard and ChatGPT-3.5 generated responses for surgical treatment of glaucoma.

🌟 Stay Updated!
Join Dr. Ailexa’s channels to receive the latest insights in health and AI.

⚡ Quick Summary

A recent study evaluated the appropriateness and readability of responses generated by ChatGPT-3.5 and Google Bard regarding the surgical treatment of glaucoma. The findings revealed that while ChatGPT-3.5 provided more accurate information, Google Bard’s responses were significantly easier to read.

🔍 Key Details

  • 📊 Study Design: Retrospective, cross-sectional study
  • 🔍 Participants: 25 common questions related to glaucoma surgery
  • 👨‍⚕️ Evaluators: Glaucoma specialists
  • 📈 Metrics: Appropriateness and readability scores

🔑 Key Takeaways

  • 📊 Accuracy: ChatGPT-3.5 provided appropriate answers 96% of the time.
  • 📉 Google Bard: Achieved 68% appropriateness in responses.
  • 📝 Readability: Google Bard had a Flesch Reading Ease Score of 57.6, indicating easier comprehension.
  • 📚 ChatGPT-3.5: Had a lower readability score with a Flesch Reading Ease Score of 22.6.
  • 🔍 Sentence Length: Google Bard had fewer long sentences compared to ChatGPT-3.5.
  • 🌐 Real-time Data: Google Bard is trained with real-time data, offering more current information.
  • 💡 Importance: Clear and accurate online information is crucial for glaucoma patients.
  • 🔄 Future Research: Further studies are needed to evaluate these AI tools across various medical topics.

📚 Background

As patients increasingly turn to AI chatbots for medical information, the need for clear and accurate content becomes paramount. Glaucoma, a leading cause of blindness, requires patients to make informed decisions about their treatment options. This study highlights the variability in the quality and readability of online information, emphasizing the role of AI in patient education.

🗒️ Study

Conducted by Ichhpujani et al., this study aimed to assess the appropriateness and readability of responses generated by two AI platforms, ChatGPT-3.5 and Google Bard, concerning surgical treatments for glaucoma. A total of 25 questions were posed, and responses were evaluated by glaucoma specialists for their accuracy and clarity.

📈 Results

The results indicated that ChatGPT-3.5 provided a higher percentage of appropriate answers at 96%, compared to Google Bard’s 68%. However, Google Bard’s responses were significantly easier to read, with a Flesch Reading Ease Score of 57.6 versus ChatGPT-3.5’s 22.6. This suggests that while ChatGPT-3.5 is more accurate, its complexity may hinder comprehension for the general public.

🌍 Impact and Implications

The findings of this study have significant implications for patient education and health literacy. As more individuals seek information online, the ability of AI tools to provide accessible and accurate information is crucial. This research underscores the importance of verifying the clarity and accuracy of online resources that patients rely on for their health decisions.

🔮 Conclusion

This study highlights the dual role of AI in healthcare: while ChatGPT-3.5 excels in accuracy, Google Bard offers greater readability. As we navigate this exciting new frontier in patient education, it is essential to ensure that the information provided is both accurate and comprehensible. Continued research in this area will be vital for enhancing health literacy among patients.

💬 Your comments

What are your thoughts on the use of AI in providing medical information? Do you believe readability is as important as accuracy? Let’s start a conversation! 💬 Leave your thoughts in the comments below or connect with us on social media:

Appropriateness and readability of Google Bard and ChatGPT-3.5 generated responses for surgical treatment of glaucoma.

Abstract

AIM: To evaluate the appropriateness and readability of the medical knowledge provided by ChatGPT-3.5 and Google Bard, artificial-intelligence-powered conversational search engines, regarding surgical treatment for glaucoma.
METHODS: In this retrospective, cross-sectional study, 25 common questions related to the surgical management of glaucoma were asked on ChatGPT-3.5 and Google Bard. Glaucoma specialists graded the responses’ appropriateness, and different scores assessed readability.
RESULTS: Appropriate answers to the posed questions were obtained in 68% of the responses with Google Bard and 96% with ChatGPT-3.5. On average, the responses generated by Google Bard had a significantly lower proportion of sentences, having more than 30 and 20 syllables (23% and 52% respectively) compared to ChatGPT-3.5 (66% and 82% respectively), as noted by readability. Google Bard had significantly (p<0.0001) lower readability grade scores and significantly higher "Flesch Reading ease score", implying greater ease of readability amongst the answers generated by Google Bard. DISCUSSION: Many patients and their families turn to LLM chatbots for information, necessitating clear and accurate content. Assessments of online glaucoma information have shown variability in quality and readability, with institutional websites generally performing better than private ones. We found that ChatGPT-3.5, while precise, has lower readability than Google Bard, which is more accessible but less precise. For example, the Flesch Reading Ease Score was 57.6 for Google Bard and 22.6 for ChatGPT, indicating Google Bard's content is easier to read. Moreover, the Gunning Fog Index scores suggested that Google Bard's text is more suitable for a broader audience. ChatGPT's knowledge is limited to data up to 2021, whereas Google Bard, trained with real-time data, offers more current information. Further research is needed to evaluate these tools across various medical topics. CONCLUSION: The answers generated by ChatGPT-3.5™ AI are more accurate than the ones given by Google Bard. However, comprehension of ChatGPT-3.5™ answers may be difficult for the public with glaucoma. This study emphasized the importance of verifying the accuracy and clarity of online information that glaucoma patients rely on to make informed decisions about their ocular health. This is an exciting new area for patient education and health literacy.

Author: [‘Ichhpujani P’, ‘Parmar UPS’, ‘Kumar S’]

Journal: Rom J Ophthalmol

Citation: Ichhpujani P, et al. Appropriateness and readability of Google Bard and ChatGPT-3.5 generated responses for surgical treatment of glaucoma. Appropriateness and readability of Google Bard and ChatGPT-3.5 generated responses for surgical treatment of glaucoma. 2024; 68:243-248. doi: 10.22336/rjo.2024.45

Share on facebook
Facebook
Share on twitter
Twitter
Share on linkedin
LinkedIn
Share on whatsapp
WhatsApp

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.