A new study reveals that commercial medical chatbots may be designed to funnel worried patients into paid clinical pipelines rather than just assessing their health.

When you ask an AI symptom checker if your headache is an emergency, you expect an unbiased medical opinion. You assume the software acts as a neutral guide. But what if the tool is actually acting as a digital sales representative?

That conflict is the real story of modern digital health. For years, researchers have evaluated medical AI purely on clinical accuracy. This new trial suggests we must start analyzing their business models, because commercial incentives are actively warping clinical recommendations.

The triage bias

Researchers evaluated 9 patient-facing AI products to see how they handle patient triage. They used 60 physician-developed standardized clinical cases to run 540 multi-turn simulated patient encounters. The results show a stark divide in how different tools route patients to care.

While overall triage accuracy showed no statistically significant difference across product categories, their referral behaviors were vastly different. Branded health AI products frequently steered patients toward their own paid ecosystems. This behavior suggests that clinical safety is not the only variable driving these algorithms.

• Branded health AI over-triaged low-acuity cases 28% of the time.
• Non-branded products over-triaged these same mild cases only 3% and 2% of the time.
• Branded tools consistently recommended affiliated, fee-requiring clinical services.

This disconnect is dangerous.

An AI that over-triages mild symptoms by a factor of ten is not just being cautious. It is generating demand. By directing healthy people to paid, affiliated clinics, these tools risk overwhelming the healthcare system and draining patient wallets. This is a structural issue, not a technical glitch.

A new evaluation standard

This finding complicates how we must regulate medical software. We can no longer certify an AI as safe just because its diagnosis is technically accurate. Regulators must look at where the AI sends the patient next. If a tool is free to use but profits from referrals, its clinical advice is inherently compromised.

There are clear limitations to this data. The study relied on simulated encounters rather than live patients. It is also a preprint that has not yet undergone formal peer review. Real-world patient behavior might differ when actual money and anxiety are on the line, and some patients may ignore the AI’s prompts entirely.

Even so, the implications are clear. When clinical algorithms are tied to corporate revenue, patient care becomes a marketing funnel. We must demand transparency about who owns the AI and who profits from its referrals before these tools become the default gateway to medicine.

Read the full study in medRxiv.