Is AI Coaching Actually Effective? What the Research Says

“Does AI coaching actually work?” is a fair question, and one that the wellness industry has not always answered honestly. This piece walks through what the research has and hasn’t shown, where the evidence is strong, where it’s thin, and how to read a digital-mental-health study with the skepticism it deserves.

The honest short answer

Digital cognitive behavioral therapy, the broader category that AI coaching is the latest evolution of, has a robust evidence base for mild-to-moderate depression and anxiety in non-clinical and subclinical populations. AI conversational coaching specifically has early- stage evidence for symptom reduction and engagement, with smaller studies than what therapy can claim. The honest summary: meaningful but modest effects, in narrowly-bounded populations, with study designs that have known limitations.

That summary is unsatisfying compared to marketing copy, and it’s unsatisfying for a reason: the field is young, the products change every six months, and the tools we use to evaluate them weren’t built for software that updates between Tuesday and Thursday. The right posture for a reader is to take the evidence seriously and discount the marketing.

What “effective” even means

Before any number is meaningful, you have to be specific about the question. Three questions masquerade as one in this space, and they have different answers.

Does it reduce symptoms? Most digital-mental-health studies measure changes on validated symptom scales, the PHQ-9 for depression, the GAD-7 for anxiety, sometimes the DASS-21 or Kessler-10 for general distress. A reduction of around 5 points on the PHQ-9 is the typical threshold for “clinically meaningful” in depression research, though the exact number varies by study and population.¹

Does it improve function? Symptom scores are easier to measure than the actual things people care about: sleep quality, work performance, relationship satisfaction, the felt sense of being okay. Some studies include functional measures; many don’t.

Does it persist? Most digital-mental-health trials run 4-12 weeks. Few follow up at six months. Even fewer follow up at a year. The question of whether effects last beyond the study period, and whether users continue to engage with the product long enough to maintain effects, is largely open.

Holding all three of these in your head while reading a research claim is most of the work.

The digital CBT evidence base

Digital CBT, internet-delivered CBT, computerized CBT, or whatever label is in fashion, has been studied since the early 2000s. The evidence base, by 2026, is large and mostly positive for adult populations with mild-to-moderate depression and anxiety.

Andrews and colleagues’ 2018 meta-analysis pooled 64 RCTs and reported moderate effect sizes for digital CBT versus waitlist controls, Cohen’s d around 0.8 for depression and around 0.8 for anxiety, which is in the range of effect sizes reported for face-to-face CBT.² Subsequent meta-analyses have continued to support the basic finding while adding nuance about who benefits most (motivated users with mild-to-moderate symptoms), what features matter (some clinician contact appears to boost effects), and what doesn’t generalize (severe depression, complex trauma, populations with low digital literacy).

The two large caveats. One: most of these studies compare digital CBT to a waitlist, which is a low bar. Comparisons to active controls (face-to-face therapy, treatment-as-usual) tend to show smaller, though still positive, effects.³ Two: completion rates in digital CBT trials are often low. Forty to sixty percent of users drop out before completing the protocol, which means the headline effect size is largely on people who finished, a self-selected, motivated subset.

Conversational AI specifically

AI coaching products use a different delivery format than the structured-module digital CBT of the 2010s. Instead of clicking through a curriculum, the user has a conversation. The evidence base for this specific format is smaller and younger.

The most-cited early study is Fitzpatrick, Darcy, and Vierhile’s 2017 RCT of Woebot, a rule-based conversational agent (not a generative model, this distinction matters). Across 70 college students with depression or anxiety symptoms, two weeks of Woebot use produced significant reductions in PHQ-9 scores compared to an information-only control.⁴ It was a small, short, single-population study, but it was the first peer-reviewed demonstration that a conversational agent could meaningfully shift symptoms.

Subsequent work has extended the evidence base in several directions. A 2022 meta-analysis of chatbot-based mental-health interventions pooled fifteen studies and reported significant improvements on depression and distress measures, with smaller effects on anxiety and well-being.⁵ Wysa has accumulated its own evidence base, including effectiveness data for users with chronic pain plus depression and a workplace-population study showing engagement and symptom improvement.

Two important context points. One: most of these studies were conducted on rule-based or retrieval-augmented systems, not the generative-AI products that proliferated after 2023. The evidence on whether large-language-model-driven coaching is better, worse, or equivalent is still actively being collected. Two: the studies that exist are largely on Woebot and Wysa, the two products that invested early in research. Most of the AI wellness apps on the App Store have no peer-reviewed evidence at all.

Where AuraLift’s category sits

AuraLift is a generative-AI coaching product built for a non-clinical audience, adults who are functioning but not thriving. The relevant body of evidence is the digital CBT meta-analysis literature plus the chatbot-specific subset above, with appropriate weight given to the population mismatch (most studies are on people with subclinical or mild-to- moderate symptoms; AuraLift’s target user often sits below that threshold).

We do not claim, and will not claim, that AuraLift treats depression, anxiety, or any other clinical condition. We claim that AuraLift uses techniques drawn from CBT, DBT, ACT, and mindfulness research; that those techniques have a substantial evidence base when delivered by clinicians and a more modest evidence base when delivered digitally; and that the audience we’re built for tends to fit the populations where digital approaches show the most benefit.

That’s a more careful claim than “AI therapy that works,” and it’s the only claim the evidence supports.

How to read a digital-mental-health study

Five questions, in order, get you most of the way to a useful read of any study in this space.

1. Who’s in the sample? Studies on college students with mild symptoms tell you something different from studies on a clinical depression cohort. Read the inclusion and exclusion criteria. Studies with strict exclusions (no current therapy, no current medication, no severe symptoms) often show larger effects, because the population is narrow and motivated.

2. What’s the comparison? A “significant” effect against a waitlist control is a much weaker claim than a “significant” effect against an active comparison (face-to-face therapy, another app, treatment-as-usual). Scan for the comparison group early, it changes the meaning of the result.

3. What’s the duration? Two weeks is too short. Four-to-eight weeks is typical. Three months is good. Six-month follow-up is rare and valuable. A symptom drop at week 4 that disappears by week 12 isn’t the win it looks like.

4. What’s the dropout? Most digital studies report on “completers”, people who finished the protocol. If 60% dropped out, the reported effect is for the 40% who stuck around. The intent-to-treat analysis (everyone, completers and dropouts) is the more honest number; if it’s buried or missing, treat the headline with suspicion.

5. Who funded it? Industry-funded studies aren’t automatically suspect, but the funding source belongs in your read. Independent academic replication is much rarer in this space than it should be.

Red flags in marketing claims

• “Clinically proven” with no citation. Real claims cite studies; marketing claims paraphrase them.
• Effect-size language without context. “76% of users improved” tells you nothing without baseline severity, comparison group, and duration.
• Symptom-treatment claims for products that aren’t cleared or regulated as treatments. The FDA has a category for digital therapeutics (de novo 510(k) clearance for prescription digital therapeutics is the main path); a consumer wellness app that hasn’t gone through it shouldn’t claim treatment outcomes.
• “Backed by science” as a phrase. Everything is backed by something; the question is what, by whom, and how strongly.
• Therapist comparisons: “as effective as therapy” or “replaces your therapist.” This is the marketing claim most likely to draw regulatory attention, and most likely to be false.
• Diagnosis output. A product that issues a diagnosis based on a brief conversation, even one couched as “you might have...”, is doing something it shouldn’t.

What the field still needs

Three things would meaningfully improve the evidence picture.

Studies on the actual products people use. The bulk of the evidence base was generated on rule-based or retrieval-augmented systems through 2022. The generative-AI coaching products that dominate the App Store today have far less peer-reviewed support because the products move faster than the trials can complete. Closing this gap will require AI coaching companies to invest in research as a category, not as a marketing exercise.

Longer follow-up. Most studies stop at 4-12 weeks. We don’t know what happens at six months or a year, and the answer matters more for products people use continuously than for products built around a finite curriculum.

Studies on populations that aren’t college students. A non-trivial share of the literature is on college students because they’re cheap to recruit and easy to consent. The audience for AI wellness coaching skews older, more diverse, and harder to recruit, and the effects in that audience may differ.

A practical bottom line

If you’re evaluating an AI wellness coaching product, the evidence-grounded take is:

• The underlying techniques, CBT, DBT, ACT, mindfulness, have strong evidence bases when delivered by clinicians.
• The digital delivery of those techniques (digital CBT) has a robust evidence base for mild-to-moderate symptoms in motivated users.
• The conversational-AI delivery format has early but credible evidence for the same population, with effect sizes in the range of digital CBT’s.
• The evidence is thinnest for severe symptoms, complex presentations, and long-term maintenance, which is why coaching products should not be marketed as treatment for clinical conditions.
• The right way to use a coaching product is for what it’s built for: lightweight, accessible, between-session-style support for adults who are functional but want a place to reflect with structure. For anything heavier, see a clinician.

If you want the longer treatment of where the boundary sits between coaching and clinical care, see AI Coach vs Therapist. If you want to know what AuraLift specifically is and isn’t doing, see What an AI Coach Can and Cannot Do.