A new study reveals that simply telling an AI to act like an insurance company causes it to reject treatments that human doctors deem necessary.

If you ask a medical AI for advice, who is actually answering? It turns out the answer depends entirely on the persona you assign it. By changing a single line in a prompt, developers can shift an AI from a compassionate caregiver to a cost-cutting bureaucrat.

This malleability is a massive liability for healthcare. It means these models have no stable ethical core. They do not think like doctors. Instead, they merely mirror the incentives of whoever writes the prompt.

How the models drifted

Researchers tested three frontier models: Claude Opus 4.6, GPT-5.4, and Gemini 3.1 Pro. They ran 25 ethically complex medical cases through the models across three stakeholder perspectives: physician, patient, and insurer. Out of 675 total responses, the researchers compared the AI decisions against a benchmark panel of six human physicians.

When programmed to think like an insurer, the models quickly abandoned patient care. The shift in alignment was stark:

• GPT-5.4 reduced its alignment with human physicians by 50%.
• Gemini 3.1 Pro dropped its alignment by 45%.
• Claude Opus 4.6 showed a 10.5% drop, which was not statistically significant.
• Across all models, the insurer prompt shifted the primary ethical value from beneficence (27%) to financial stewardship (20%).

The illusion of objectivity

This is not just about a drop in accuracy. It exposes a deeper flaw in how we evaluate medical AI. We often treat these models as objective calculators of clinical data. In reality, they are ethical chameleons.

If an insurance company deploys these models to review claims, the AI will naturally prioritize saving money over saving lives. It does this not because it understands economics, but because it is mimicking a cost-cutter. This creates a highly automated, polite, and systematic way to deny necessary care.

We must be honest about the limits of this research. This was a simulation using 25 hypothetical cases, not real-time clinical workflows. The study also relied on a small panel of six physicians as the gold standard. Different doctors might have different consensus points.

Even so, the warning is clear. We cannot trust AI to make clinical decisions without strict guardrails. If a simple prompt can cut patient-centric decisions in half, then the software is too fragile—and too easily manipulated—to run without human oversight.

Read the full study in medRxiv.