Feeding raw medical data directly into clinical AI models might be degrading their performance without anyone noticing.

Engineers often assume that advanced language models can digest raw clinical data in any format. We pour billions into building smarter neural networks, expecting them to parse messy, nested JSON structures natively. A new benchmark on UK health data reveals this assumption is wrong.

How you format Fast Healthcare Interoperability Resources (FHIR) data before feeding it to an AI drastically alters its accuracy. This is not a minor coding detail. It is a hidden lever that can make or break clinical safety.

The formatting sweet spot

The study introduced FHIRBench-UK to evaluate how different data structures affect performance. Researchers tested 5 large language models across 6 serialization formats and 3 clinical tasks. Using 100 UK Core FHIR patient bundles, they generated 18,000 scored prompts across clean and perturbed cohorts.

The format choice had a massive, statistically significant impact on clinical quality (Kruskal-Wallis H=163.86, p<10^-33, with a 0.24 delta on a 5-point scale). Crucially, the best format depends entirely on the task at hand:

• raw_json dominates for clinical QA but is suboptimal in 58% of all other scenarios.
• hybrid_adaptive yields the best results for complex clinical reasoning.
• structured_markdown is the optimal choice for patient summarization.

This variance challenges the lazy deployment pattern of dumping raw JSON into a prompt window. It proves that data preprocessing is just as vital as model size.

A free performance boost

The implications for resource-constrained healthcare systems like the NHS are immediate. Upgrading to the largest proprietary models is expensive and raises data privacy concerns. This research offers a cheaper alternative.

Model capability heavily moderates format sensitivity. While a premium model like Claude Sonnet 4.5 showed a mere 0.10-point sensitivity to formatting, the mid-tier Llama 3.3 swung by 0.39 points. For budget-constrained deployments, implementing task-aware serialization routing acts as a zero-cost quality intervention. It allows cheaper, open-weight models to punch far above their weight class.

The dangerous metric trap

Perhaps the most alarming finding is how we measure AI success. The study confirmed a complete ranking inversion between token-level F1 scores and actual clinical quality, with a correlation of rho = -0.90.

This means the metrics engineers traditionally use to evaluate language models run opposite to clinical safety. An AI output that looks highly accurate on a standard NLP spreadsheet is often the most clinically flawed. We must abandon generic text metrics if we want to deploy these tools safely in hospitals.

Of course, this study has limitations. It is a preprint focused on UK Core FHIR profiles. While the findings replicated under simulated data perturbation, real-world clinical environments contain messier, unstructured narratives that this benchmark did not fully capture.

Even so, the message is clear. Stop waiting for smarter models to solve data ingestion. Smart routing of data formats is the easiest way to make clinical AI safer today.

Read the full study on medRxiv.