AI and Mental Health — The Promise, the Risk, and What Must Change

By late 2025, OpenAI reported that 1.2 million people per week were using ChatGPT to discuss suicide. Not as a statistic to be alarmed by in the abstract — but as a measure of how deeply AI systems have embedded themselves in the most vulnerable moments of human experience. ^[1]

That number demands a serious response from every technology leader building or deploying AI at scale. What is happening when those 1.2 million conversations take place? What happens next? And are the systems involved equipped for the weight of that responsibility?

The evidence, as it stands, is deeply concerning.

What the research actually shows

In August 2025, a peer-reviewed study published in Scientific Reports tested 29 AI chatbots against standardized suicide-risk scenarios based on the Columbia-Suicide Severity Rating Scale. Not a single one provided an adequate response. ^[2]

A February 2026 study from Mount Sinai's Icahn School of Medicine, published in Nature Medicine, evaluated ChatGPT Health in controlled clinical scenarios. Separately, research from McGill University found that publicly reported harm related to AI chatbot use most frequently involved death by suicide — and that fatal outcomes were disproportionately represented in minors. ^[3]

The challenge is not just about direct crisis responses. Research from King's College London published in JMIR Mental Health in 2026 found that risk often does not emerge at a single tipping point but accumulates across extended dialogue — and that most safety evaluation approaches are entirely misaligned with this reality. ^[4] A chatbot can pass a one-turn safety test and still cause serious harm across a 30-turn conversation.

The promise is real — and that makes the stakes higher

It would be intellectually dishonest to argue that AI has no role in mental healthcare. The global mental health workforce shortage is acute and structural. There are simply not enough trained clinicians — anywhere in the world — to meet current demand. In the Gulf specifically, mental health conditions in the Middle East represent a growing share of total disease burden, while stigma, insufficient specialist capacity, and uneven service availability create significant barriers to care. ^[5]

A 2025–2026 real-world pilot of a purpose-built mental health generative AI, published in a peer-reviewed study, showed meaningful reductions in depression and anxiety symptoms across 305 users over 10 weeks — when the system was specifically designed and clinically validated for that purpose. ^[6]

This is the crucial distinction. AI built deliberately for mental health, with clinical oversight, evidence-based frameworks, and crisis escalation pathways, can be a genuine force for good. General-purpose chatbots deployed into vulnerable mental health contexts without those safeguards cannot.

I've seen this distinction up close. As co-founder of Maternally, a perinatal mental health venture, we built technology for one of the most sensitive intersections of human vulnerability and digital support — women navigating pregnancy and early motherhood, a period when mental health conditions are common and chronically underdiagnosed. The lesson we learned, repeatedly, is that building responsibly in this space is slower, harder, and more expensive than building quickly. It requires clinical partnerships, evidence-based protocols, crisis pathways, and an unwillingness to optimize for engagement at the expense of safety. It is not a feature you add. It is an architecture you commit to from the start.

What needs to change

The regulatory landscape is beginning to respond, but unevenly. A 2025 review found 143 bills related to AI and mental health introduced across US states between 2022 and 2025. As of May 2025, only 11 states had enacted meaningful legislation. Families of teenagers who died by suicide after chatbot interactions testified before Congress. Pennsylvania sued Character.AI for presenting its bots as licensed medical professionals. The FTC has opened inquiries into multiple platforms. ^[7]

But regulation alone is insufficient. Here is what technology leaders building or deploying AI systems should be asking — and demanding — right now:

Has our system been tested against multi-turn crisis scenarios, not just single-prompt safety checks? Do we have a validated crisis escalation pathway that connects distressed users to human support? Have we specifically evaluated performance with vulnerable populations — adolescents, individuals with serious mental illness, those in acute crisis?

— Questions every AI product leader should be answering before deployment

Specific recommendations

For AI developers: Adopt the VERA-MH framework — an open-source, clinician-informed evaluation standard specifically built for mental health safety assessment in AI. ^[8] Make multi-turn safety evaluation standard, not optional. Never allow a general-purpose chatbot to present itself as a therapist or clinical resource.

For enterprise deployers: If you are deploying AI systems that employees or customers might use for emotional support — and they will, whether you design for it or not — build explicit mental health safety guardrails and crisis referral pathways into your deployment. Assume the most vulnerable user, not the average one.

For Gulf governments and health authorities: The region's high AI adoption rate and significant mental health stigma create a specific risk profile. Young people in the Gulf are turning to AI chatbots precisely because human mental health support feels inaccessible or stigmatized. ^[5] This creates an urgent need for region-specific AI mental health safety standards — not borrowed wholesale from US or European regulatory frameworks, but built for the cultural, linguistic, and clinical realities of this population.

For investors: Purpose-built mental health AI, developed with clinical rigor and safety-first architecture, represents both a genuine commercial opportunity and a moral imperative. The market is large, the need is acute, and the current alternatives — under-resourced human services and unsafe general-purpose chatbots — are failing people. Back the builders who are doing this correctly.

The bottom line

AI will be part of the mental health landscape. That is not a forecast — it is already the reality for millions of people who are turning to these systems in their most vulnerable moments, often because there is nowhere else to turn. The question is whether the technology industry will take that responsibility seriously enough to build for it properly, regulate for it honestly, and measure its failures as rigorously as it measures its growth.

The standard cannot be "we tried to be helpful." When someone in crisis reaches out to an AI system, the standard has to be "we were ready."