Overview

A recent study from the University of Colorado Anschutz Medical Campus reveals that free, open-source artificial intelligence (AI) tools can assist doctors in reporting medical scans with accuracy comparable to that of expensive commercial systems, all while safeguarding patient privacy.

Key Findings

• The study was published in the journal npj Digital Medicine.
• Open-source AI tools provide a cost-effective alternative to commercial options like ChatGPT, which often require sensitive data to be sent to external servers.
• Lead author Aakriti Pandita, MD, emphasized that hospitals can achieve accurate results without relying on costly or privacy-compromising AI systems.

Importance of Structured Reporting

Doctors typically create notes or free-text reports when analyzing medical scans, such as ultrasounds. While these notes are valuable, they may not always meet the structured format needed for various clinical applications. Structured data aids hospitals in:

1. Tracking patient outcomes
2. Identifying trends
3. Conducting research more efficiently

Privacy Concerns with Advanced AI Systems

Many advanced AI systems, including GPT-4 from OpenAI, necessitate the transmission of patient data over the internet, raising significant privacy concerns in healthcare settings.

Study Methodology

The research team focused on thyroid nodules, which are often detected during ultrasounds. They utilized a scoring system called ACR TI-RADS to assess the cancer risk of these nodules. To train the AI models without using real patient data, they generated:

• 3,000 synthetic radiology reports that mimicked medical language without containing private information.
• Six different free AI models were trained to read and score these reports.

Performance Comparison

The models were tested against 50 real patient reports from a public dataset, with results compared to commercial AI tools like GPT-3.5 and GPT-4. Notably:

• The open-source model Yi-34B matched the performance of GPT-4 with minimal training.
• Smaller models outperformed GPT-3.5 in certain tests.

Advantages of Open-Source AI Tools

According to Nikhil Madhuripan, MD, senior author of the study, commercial tools, while powerful, are often impractical in healthcare due to their costs and privacy risks. In contrast, open-source AI tools:

• Can operate within a hospital’s secure system, ensuring sensitive information remains protected.
• Eliminate the need for expensive GPU clusters.

Future Applications

The study suggests that synthetic data can effectively train AI tools, paving the way for affordable, customized AI systems across various healthcare domains. The research team envisions that similar tools could eventually assist in:

• Reviewing CT reports
• Organizing medical notes
• Monitoring disease progression

Aakriti Pandita concluded, “This isn’t just about saving time; it’s about creating AI tools that are practical for everyday medical use without excessive costs or privacy compromises.”