โก Quick Summary
This study explored how large language models (LLMs) can assist in generating health guideline questions, specifically for the 2024 ARIA guidelines on allergic rhinitis. The findings revealed that LLMs can complement traditional methods, with 6 out of 39 final questions being novel contributions from the AI.
๐ Key Details
- ๐ Queries Analyzed: 3,975 unique queries retrieved from Google Trends
- ๐งฉ Questions Identified: 37 relevant questions derived from search queries
- โ๏ธ AI Technology: Large Language Models (LLMs)
- ๐ Prioritized Questions: 6 questions not previously considered by the panel
๐ Key Takeaways
- ๐ค LLMs can effectively generate relevant health guideline questions.
- ๐ก Novel insights were provided by AI, with 6 questions prioritized that were not initially considered.
- ๐ Complementary Approach: LLMs can enhance traditional expert-driven methods.
- ๐ Study Context: Focused on allergic rhinitis and its impact on asthma.
- ๐ Google Trends was utilized to identify public interest in specific health queries.
- ๐ Future Implications: Potential for broader applications in health guideline development.
๐ Background
The development of health guidelines typically relies on expert opinions and established practices. However, the integration of artificial intelligence into this process presents an opportunity to enhance the relevance and comprehensiveness of guideline questions. By leveraging public interest data and AI capabilities, researchers aim to bridge the gap between expert knowledge and patient needs.
๐๏ธ Study
This study focused on the 2024 ARIA guidelines for allergic rhinitis, employing two main approaches for generating guideline questions. The first involved analyzing popular online search queries related to allergic rhinitis using Google Trends, while the second utilized LLMs to directly generate questions from the perspectives of patients and clinicians.
๐ Results
The analysis yielded 3,975 unique queries, from which 37 relevant questions were identified. Notably, 22 of these questions had not been previously posed by guideline panel members, and 4 were prioritized for inclusion in the final guidelines. Additionally, direct interactions with LLMs resulted in the generation of 22 unique questions, with 11 being new contributions to the panel’s considerations.
๐ Impact and Implications
The findings from this study suggest that LLMs can play a significant role in the development of health guidelines, offering a fresh perspective that complements traditional methods. By integrating AI into the guideline development process, we can ensure that the questions posed are not only relevant to experts but also resonate with the concerns of patients and the public. This could lead to more effective and patient-centered healthcare practices.
๐ฎ Conclusion
This research highlights the transformative potential of AI in the realm of health guideline development. By utilizing LLMs, researchers can uncover valuable insights that may have otherwise been overlooked. As we move forward, it is essential to continue exploring the integration of AI technologies in healthcare to enhance the quality and relevance of clinical guidelines.
๐ฌ Your comments
What are your thoughts on the use of AI in developing health guidelines? We would love to hear your insights! ๐ฌ Share your comments below or connect with us on social media:
Artificial Intelligence-Supported Development of Health Guideline Questions.
Abstract
BACKGROUND: Guideline questions are typically proposed by experts.
OBJECTIVE: To assess how large language models (LLMs) can support the development of guideline questions, providing insights on approaches and lessons learned.
DESIGN: Two approaches for guideline question generation were assessed: 1) identification of questions conveyed by online search queries and 2) direct generation of guideline questions by LLMs. For the former, the researchers retrieved popular queries on allergic rhinitis using Google Trends (GT) and identified those conveying questions using both manual and LLM-based methods. They then manually structured as guideline questions the queries that conveyed relevant questions. For the second approach, they tasked an LLM with proposing guideline questions, assuming the role of either a patient or a clinician.
SETTING: Allergic Rhinitis and its Impact on Asthma (ARIA) 2024 guidelines.
PARTICIPANTS: None.
MEASUREMENTS: Frequency of relevant questions generated.
RESULTS: The authors retrieved 3975 unique queries using GT. From these, they identified 37 questions, of which 22 had not been previously posed by guideline panel members and 2 were eventually prioritized by the panel. Direct interactions with LLMs resulted in the generation of 22 unique relevant questions (11 not previously suggested by panel members), and 4 were eventually prioritized by the panel. In total, 6 of 39 final questions prioritized for the 2024 ARIA guidelines were not initially thought of by the panel. The researchers provide a set of practical insights on the implementation of their approaches based on the lessons learned.
LIMITATION: Single case study (ARIA guidelines).
CONCLUSION: Approaches using LLMs can support the development of guideline questions, complementing traditional methods and potentially augmenting questions prioritized by guideline panels.
PRIMARY FUNDING SOURCE: Fraunhofer Cluster of Excellence for Immune-Mediated Diseases.
Author: [‘Sousa-Pinto B’, ‘Vieira RJ’, ‘Marques-Cruz M’, ‘Bognanni A’, ‘Gil-Mata S’, ‘Jankin S’, ‘Amaro J’, ‘Pinheiro L’, ‘Mota M’, ‘Giovannini M’, ‘de Las Vecillas L’, ‘Pereira AM’, ‘Lityลska J’, ‘Samolinski B’, ‘Bernstein J’, ‘Dykewicz M’, ‘Hofmann-Apitius M’, ‘Jacobs M’, ‘Papadopoulos N’, ‘Williams S’, ‘Zuberbier T’, ‘Fonseca JA’, ‘Cruz-Correia R’, ‘Bousquet J’, ‘Schรผnemann HJ’]
Journal: Ann Intern Med
Citation: Sousa-Pinto B, et al. Artificial Intelligence-Supported Development of Health Guideline Questions. Artificial Intelligence-Supported Development of Health Guideline Questions. 2024; (unknown volume):(unknown pages). doi: 10.7326/ANNALS-24-00363