Overview
A recent study published on September 30, 2025, in the journal PLOS Digital Health reveals a new artificial intelligence tool capable of scanning social media to detect adverse events related to consumer health products. The research was conducted by John Ayers and his team at the University of California, San Diego.
Importance of Post-Market Surveillance
Continuous monitoring of consumer product safety is essential for public health. Current systems for reporting adverse events (AEs) rely on voluntary submissions from healthcare professionals and manufacturers to the U.S. Food and Drug Administration (FDA). With the rise of consumer health products, including cannabis-derived items and dietary supplements, there is a pressing need for innovative detection methods.
Introducing Waldo
The study evaluated the effectiveness of an automated machine learning tool named “Waldo.” This tool analyzes social media content to identify consumer reports of adverse events. Key findings include:
- Waldo was tested on Reddit posts concerning cannabis-derived products.
- It achieved an impressive accuracy rate of 99.7% when compared to human annotations.
- In a larger dataset of 437,132 Reddit posts, Waldo identified 28,832 potential harm reports.
- Manual validation showed that 86% of these were confirmed as true adverse events.
Open-Source Accessibility
The research team has made Waldo open-source, allowing researchers, clinicians, and regulators to utilize the tool for enhanced patient safety.
Expert Insights
Lead author Karan Desai emphasized the significance of online health experiences, stating, “These voices provide valuable safety signals that traditional reporting systems often overlook.” John Ayers noted that this project showcases how digital health tools can improve post-market surveillance.
Second author Vijay Tiyyala highlighted the technical advancements, mentioning that the model used, RoBERTa, outperformed leading chatbots in adverse event detection.
Conclusion
The team aims to democratize access to Waldo, promoting open science and enhancing patient safety through improved detection of health risks associated with consumer products.