Training AI on general medical textbooks actually hurts its ability to mirror expert prescribing habits.
Most clinical AI developers assume that more data always improves accuracy. They mix textbook medical knowledge with clinical case notes, hoping the model synthesizes both seamlessly. A new study on Traditional Chinese Medicine (TCM) shows this approach backfires when trying to replicate specialized expert care.
That friction exposes a major flaw in general clinical AI. When general domain knowledge conflicts with hyper-specific practitioner habits, unified training dilutes clinical precision.
Separating Theory From Practice
Researchers built DFGLM-TCM, a modular system that splits theoretical knowledge from practitioner experience. The knowledge engine drew from a 22-GB corpus with roughly 3 million structured entries, a validated graph of over 200,000 entities, and retrieval-augmented generation. Meanwhile, the experience engine learned exclusively from 5,000 authentic outpatient records from a single senior practitioner, plus 2,000 expert-reviewed augmented cases.
Rather than merging these streams into one model, the architecture kept them distinct. This design tackles a core bottleneck seen in other systems like Lingdan, which focus primarily on unified encoding of clinical reasoning. By decoupling theory from individual practice, the system avoids forcing general rules onto unique prescribing styles.
Performance Breakdown
The split architecture outperformed combined training methods across benchmark tests.
- On 100 standardized TCM knowledge questions, the system scored an expert-rated mean of 4.48 ± 0.76, the highest among all evaluated models.
- In 454 independent pulmonary-nodule cases, the experience module scored 8.95 ± 0.63 for prescription consistency with the master practitioner.
- Adding general TCM knowledge back into that same experience module dropped its prescription consistency score down to 7.52 ± 0.44.
- A one-month field assessment across 35 physicians at 3 primary-care institutions confirmed positive short-term usability and adoption.
Limits to Clinical Proof
High prescription consistency is a proxy for practitioner alignment, not a guarantee of superior patient outcomes. Replicating a senior doctor’s prescription patterns does not prove the treatments improved recovery or reduced side effects. Furthermore, the experience module reflects the style of just one senior expert focused on pulmonary nodules, limiting immediate generalization to other specialties.
This design shifts how health systems should deploy specialized medical AI. Similar to recent efforts like TCMChat, the key to safe deployment is architecture, not parameters. Developers building clinical reference tools should stop dumping textbooks into expert systems. Keeping general background knowledge separated from specialized diagnostic modules yields far safer, more predictable decision support.
Read the full paper in Chinese Medicine.
