A new AI tool proves that human error, not machine limitation, is the biggest obstacle to tracking missing clinical trial data.

Why do billions of dollars in medical research simply vanish from the public record? We often blame researchers for failing to publish their findings. But the truth is often much simpler: we are just terrible at keeping track of the paperwork. Clinical trials get registered, papers get published, but the two systems rarely talk to each other.

For years, the scientific community assumed human auditing was the gold standard for policing this transparency gap. A new preprint evaluating an AI tool called TrialScout upends that assumption. The algorithm did not just match trials faster than humans. It proved that human auditors were the ones making the mistakes.

This disconnect changes how we must judge scientific transparency. If our manual tracking systems are this sloppy, then policy decisions about research funding and drug safety are being made using dirty data.

Machines beat human auditors

Researchers built TrialScout to match clinical trials registered on ClinicalTrials.gov with result publications indexed in PubMed. When tested against previous human-coded datasets, the AI achieved a sensitivity of 92.5% and a specificity of 81.2%. These numbers are strong, but they do not tell the whole story.

The real surprise came when researchers analyzed the 200 cases where the AI and humans disagreed. A manual review revealed that 61.5% of those disagreements (123 out of 200) were caused by human errors, not machine failures. The human researchers had simply missed the matches.

This finding suggests that our reliance on human-curated databases has created a false sense of security. Human curators are the weak link in tracking clinical trial transparency.

Mapping the missing data

• TrialScout analyzed a random sample of 9,600 completed or terminated trials on ClinicalTrials.gov.
• It successfully located published results for 6,110 of those trials, representing a 63.6% match rate.
• Human error accounted for the majority of discrepancies in the disputed sample, with a 95% confidence interval of 54.4% to 68.3%.

This is a systemic issue with real-world consequences. When clinical trial results remain unlinked, systematic reviewers and clinical practitioners cannot find the evidence they need to treat patients safely. Automated language models are no longer just a cheap alternative to human labor. They are a necessary upgrade for accuracy.

The limits of automation

The tool still has clear boundaries. The researchers note that estimating TrialScout’s true accuracy is difficult because a perfect, error-free gold standard dataset does not exist. The tool also relies entirely on PubMed and ClinicalTrials.gov. Trials registered on other registries or published in niche journals might still slip through the cracks.

Even with these limitations, the shift is clear. We can no longer treat human curation as the benchmark for clinical data tracking.

Read the full preprint on medRxiv.