A new model bypasses rigid labels to turn raw cardiac waveforms directly into human-readable clinical narratives.

For decades, automated ECG machines have spat out the same rigid, one-sentence labels. If a patient had a complex mix of conditions, the machine fell silent or threw a generic error code. Now, researchers have bypassed this bottleneck by treating cardiac waves not as math, but as a language.

This shift challenges the very structure of digital cardiology. Instead of building separate, narrow classifiers for every single heart condition, we can now use a single model that reads raw waveforms and answers diverse clinical questions. It suggests that the future of diagnostics lies in unified foundation models that can chat about a patient’s heart.

Translating waves to words

The model, called DeepECG-Tok, uses a specialized tokenizer to map 12-lead waveforms into discrete tokens that a large language model can understand. This approach was trained on a massive dataset of 7.27 million question-answer pairs. The system does not just label diseases. It predicts complex clinical endpoints like left ventricular ejection fraction, structural heart disease, and atrial fibrillation risk.

The raw diagnostic power of this method is remarkably high. In testing, the frozen tokenizer achieved a macro-averaged AUROC of 0.96 across 77 different conditions. When tested on four external patient groups without any retraining, the endpoint classifiers maintained high accuracy with AUROCs between 0.88 and 0.90.

This level of adaptability is rare for cardiac AI. Typically, algorithms trained in one hospital system fail when deployed in another due to differences in equipment and patient demographics.

The external drop-off

But the transition from structured data to free-text reasoning reveals a stark performance gap. While the raw classification remained strong, the model’s overall reasoning score dropped significantly when tested outside its home institution.

An AI judge, which showed a high agreement score of 0.82 against two human cardiologists, evaluated the model’s performance across different environments.

• The model scored 0.71 on internal data.
• Its performance fell to between 0.50 and 0.53 on external cohorts.
• In a blinded study, human doctors rated the AI’s reports as comparable to human-written ones, with a win-tie-loss ratio of 32:35:33.
• Cardiologists showed a forced-choice preference of 0.53 for the model among decided cases.

This disconnect is the real story.

The model writes reports that sound highly convincing to human cardiologists, yet its objective reasoning accuracy drops by nearly 30% on external data. If clinicians rely on the polished, natural tone of these reports, they might miss underlying diagnostic errors.

Why this matters

This performance split exposes a critical tension in medical AI. It is easy to generate clinical-sounding prose, but much harder to maintain diagnostic accuracy across different hospital systems. For healthcare systems looking to adopt generative AI, this finding is a warning. Polished language can mask a drop in clinical accuracy.

The real challenge moving forward is not making AI sound like a doctor. It is ensuring its reasoning remains robust when it leaves the lab.

This analysis is based on research published in medRxiv.