Giving consumer smart ring data to a raw language model to spot depression fails completely, unless you teach it how to think first.

Can a language model read your sleep patterns and tell if you are depressed? Tech enthusiasts want us to believe these models can diagnose us out of the box. A new study using Oura Ring data reveals a massive flaw in that assumption.

If you feed raw wearable data into a zero-shot AI, it cheats. It looks at the baseline population, notices most people are healthy, and simply guesses “not depressed” every time. It looks highly accurate on paper but is clinically useless. This disconnect is the real story. For years, developers assumed that throwing more data at larger models would naturally yield clinical insights, but this trial suggests that the active ingredient is how we structure the prompt.

The Finnish cohort test

Researchers tested this by pulling 4 to 8 days of wearable data from 1,285 participants in the Northern Finland Birth Cohort 1986. They converted metrics like heart rate, heart rate variability, sleep, activity, and demographics into text prompts. Then they fed these prompts to three lightweight models: Llama 3.1, BioMistral, and Qwen 2.5.

The zero-shot models failed to identify the actual depressed patients. Only when researchers used few-shot prompting—giving the AI just four real-world examples to learn from in-context—did the models wake up. This simple change forced the AI to understand the patterns of the minority class rather than just guessing the safest statistical average.

How the models performed

The study compared the language models against traditional machine learning and recurrent neural networks. Llama 3.1 emerged as the clear winner once it was given context.

• Llama 3.1 with four in-context examples achieved an overall accuracy of 0.92.
• It reached a macro-F1 score of 0.82.
• It scored 0.69 F1 specifically for detecting the positive class of depression and anxiety.

The illusion of accuracy

This finding challenges how we evaluate clinical AI. A model that always guesses “healthy” in a population where most people are healthy has high accuracy, but it helps absolutely no one. We must stop celebrating high raw accuracy scores if the model cannot find the sick patients. Prompt engineering is not just a trick to get better text; it is a clinical necessity.

The good news is that we do not need massive, expensive cloud APIs for wearable diagnostics. This study proves that lightweight, open-weight models running locally can do the job. However, the short 4 to 8 days data window is a limitation. Mental health fluctuates, and a one-week snapshot might miss broader seasonal trends. The cohort is also highly specific to Northern Finland, meaning these prompting rules might need recalibration for other populations.

Read the full preprint in medRxiv.