⚡ Quick Summary
This study explores the classification of Living Kidney Donation (LKD) experiences shared on Reddit, aiming to identify potential living donors through social media discussions. Utilizing advanced models like BERT and GPT-3.5, the research achieved impressive classification accuracies of 90.7% for GPT-3.5, highlighting the potential of AI in enhancing donor education outreach.
🔍 Key Details
- 📊 Data Source: Reddit forums discussing LKD
- 🧩 Classification Categories: Present, Past, Other
- ⚙️ Technologies Used: BERT and GPT-3.5 (ChatGPT)
- 🏆 Performance: BERT: 89.3%, GPT-3.5: 90.7%
🔑 Key Takeaways
- 💡 Living kidney donation is crucial for addressing kidney failure.
- 📱 Social media can be a valuable tool for identifying potential donors.
- 🤖 AI models like BERT and GPT-3.5 can effectively classify LKD-related content.
- 🏆 High accuracy of 90.7% for GPT-3.5 demonstrates the effectiveness of AI in this context.
- 🗣️ Dialogue until classification consensus method improves classification performance.
- 🔍 Post hoc analysis revealed sensible reasoning in model predictions.
- 🌍 Implications for donor education could significantly increase living donor numbers.
- 📈 Future research is encouraged to refine these methodologies further.
📚 Background
Living kidney donation (LKD) is a vital solution for the growing number of patients suffering from kidney failure. Despite the willingness of many individuals to donate, there remains a significant gap between the number of patients on waiting lists and the actual living donors available each year. This study aims to bridge that gap by leveraging social media discussions to identify potential living donors and direct educational interventions effectively.
🗒️ Study
The research focused on analyzing content from Reddit forums related to LKD. The authors classified the discussions into three categories: those currently dealing with LKD, those who have dealt with it in the past, and general comments about LKD. By employing a fine-tuned version of the BERT model alongside GPT-3.5, the study aimed to assess the models’ capabilities in accurately classifying these discussions.
📈 Results
The findings revealed that both BERT and GPT-3.5 achieved commendable classification accuracies of approximately 75% and 78%, respectively. However, after considering acceptable mismatched predictions, the accuracy improved significantly to 89.3% for BERT and 90.7% for GPT-3.5. This indicates that the models not only performed well but also demonstrated a capacity for contextual reasoning.
🌍 Impact and Implications
The implications of this study are profound. By utilizing AI to analyze social media discussions, we can enhance our understanding of potential living donors and tailor educational interventions accordingly. This approach could lead to an increase in living kidney donations, ultimately saving lives and improving patient outcomes. The integration of AI in healthcare discussions opens new avenues for research and donor engagement strategies.
🔮 Conclusion
This study highlights the remarkable potential of AI, particularly large language models like GPT-3.5, in classifying and understanding social media content related to living kidney donation. The innovative methods introduced, such as dialogue until classification consensus, pave the way for more effective prompt engineering in AI applications. As we continue to explore these technologies, the future of living kidney donation could be significantly transformed, leading to better outcomes for patients in need.
💬 Your comments
What are your thoughts on using social media and AI to enhance living kidney donation efforts? We would love to hear your insights! 💬 Leave your comments below or connect with us on social media:
Investigating the Classification of Living Kidney Donation Experiences on Reddit and Understanding the Sensitivity of ChatGPT to Prompt Engineering: Content Analysis.
Abstract
BACKGROUND: Living kidney donation (LKD), where individuals donate one kidney while alive, plays a critical role in increasing the number of kidneys available for those experiencing kidney failure. Previous studies show that many generous people are interested in becoming living donors; however, a huge gap exists between the number of patients on the waiting list and the number of living donors yearly.
OBJECTIVE: To bridge this gap, we aimed to investigate how to identify potential living donors from discussions on public social media forums so that educational interventions could later be directed to them.
METHODS: Using Reddit forums as an example, this study described the classification of Reddit content shared about LKD into three classes: (1) present (presently dealing with LKD personally), (2) past (dealt with LKD personally in the past), and (3) other (LKD general comments). An evaluation was conducted comparing a fine-tuned distilled version of the Bidirectional Encoder Representations from Transformers (BERT) model with inference using GPT-3.5 (ChatGPT). To systematically evaluate ChatGPT’s sensitivity to distinguishing between the 3 prompt categories, we used a comprehensive prompt engineering strategy encompassing a full factorial analysis in 48 runs. A novel prompt engineering approach, dialogue until classification consensus, was introduced to simulate a deliberation between 2 domain experts until a consensus on classification was achieved.
RESULTS: BERT and GPT-3.5 exhibited classification accuracies of approximately 75% and 78%, respectively. Recognizing the inherent ambiguity between classes, a post hoc analysis of incorrect predictions revealed sensible reasoning and acceptable errors in the predictive models. Considering these acceptable mismatched predictions, the accuracy improved to 89.3% for BERT and 90.7% for GPT-3.5.
CONCLUSIONS: Large language models, such as GPT-3.5, are highly capable of detecting and categorizing LKD-targeted content on social media forums. They are sensitive to instructions, and the introduced dialogue until classification consensus method exhibited superior performance over stand-alone reasoning, highlighting the merit in advancing prompt engineering methodologies. The models can produce appropriate contextual reasoning, even when final conclusions differ from their human counterparts.
Author: [‘Nielsen J’, ‘Chen X’, ‘Davis L’, ‘Waterman A’, ‘Gentili M’]
Journal: JMIR AI
Citation: Nielsen J, et al. Investigating the Classification of Living Kidney Donation Experiences on Reddit and Understanding the Sensitivity of ChatGPT to Prompt Engineering: Content Analysis. Investigating the Classification of Living Kidney Donation Experiences on Reddit and Understanding the Sensitivity of ChatGPT to Prompt Engineering: Content Analysis. 2025; 4:e57319. doi: 10.2196/57319