A Bridge to Care

I’ve been thinking a lot about safety for people using gen-AI chatbots for mental health. The number of weekly ChatGPT users recently surpassed 900 million. Monthly active Google Gemini users just surpassed 750 million. How many of these users are seeking psychological support or guidance? We really do not know, but Open AI has reported that 0.15% have conversations with explicit suicidal planning or intent. That is 1.35 million people at risk for suicide, each week.

In a new update on their mental health-related work, Open AI describes their efforts to increase safety. The most interesting part of this work involves adding parental controls that allow safety notifications based on their teens use of ChatGPT and potentially introducing a trusted contact feature for adults who need additional emotional support. While the idea that a therapist would contact a parent or trusted friend does not seem radical, there has long been an assumption that for social media sites, search engines, and gen-AI chatbots, “the walls don’t talk.” When a chatbot breaks through that wall to contact someone who has not been in the conversation, we’re in a new world.

What does safety actually look like when someone confides suicidal thoughts to a machine? As I said in a previous post, in the AI chatbot safety conversation thus far, there has been much focus on detection of risk and little focus on the response. Tammy Horn in an excellent AI Therapy Solutions substack post this week says, “Detection without appropriate intervention isn’t safety. It’s surveillance.” You should read her post. My poor summary follows.

Based on over 20,000 therapeutic AI conversations, Horn lists four ways in which seemingly “safe” responses actually lead to harms:

• The hard redirect. This is the pop-up to call or text a crisis line, basically saying, “I can’t help you anymore”, leaving the person less trusting.

• The topic change. This is the affirmation followed by a suggestion of a less painful topic, leaving the person less willing to discuss their plans.

• The disclaimer dump. This is the “I’m not qualified” response from a bot that usually tells someone to find mental health services that may not exist.

• The overcorrection. This is the tendency for some LLMs to over-respond once they detect a safety issue so that any personal statement triggers a safety response.

The hard redirect may be the most important because it is usually the gold standard for safety. The online safety response invokes a simple rule: if a risk of self-harm is detected, provide the user with a link to a crisis response service. This may seem like a reasonable response, but little evidence supports its value. As Horn says, “For someone who has been building emotional trust with an AI system over weeks or months, this feels like being handed a pamphlet by someone who was pretending to listen. The user learns something devastating: this system doesn’t actually understand me. It just has a tripwire. And tripwires don’t care about me — they care about liability.”

So what is the right response? Horn argues that this a clinical question not a technical spec. She is right. But we still need to figure out how a chatbot should respond in a conversation that indicates a risk for self-harm, even when that conversation meanders over several days, weeks, or months. We know what a clinician would do. An effective clinician would ask questions to learn about context, triggers, and safety planning. If the risk were serious enough, most clinicians would notify a family member and refer for hospitalization.

Should ChatGPT or Gemini, platforms that prioritize engagement, become the clinician? I hope not. They don’t have the judgment, and their companies don’t want the liability. But they could become the bridge to care. This bridge need not be a “hard redirect” or a “disclaimer dump”. It could be a series of nudges and supports to help someone get care – not a crisis service but care. Care means an ongoing relationship with a human therapist trained in the techniques that reduce suicidal thinking and behaviors.

I know, I know. We are not living in mental health Camelot. Over 169 million people (more than half the U.S. population) live in a designated Mental Health Professional Shortage Area, reports the Bureau of Health Workforce. Roughly 65% of rural counties do not have a psychiatrist. And even in counties with an appropriate workforce, the waitlist can be months for a first appointment. Are we talking about a bridge to nowhere?

When we launched Benchmark Health last year to help people find high quality mental health care, we expected to find long wait times and few clinicians. Yes, we found nine month wait times for some clinics but we also found excellent resources - some online, some brick-and-mortar - with excess capacity. Some of these services specifically focus on reducing suicidal ideation with evidence-based interventions.

Can we find high quality clinicians for 1.35 million suicidal people this week? No, but the state of care is far from hopeless. If we can find a way to shift people from the world of algorithms to the world of care, we can save lives. That 1.35 million figure is an opportunity. People have chosen to share their pain with a chatbot. The challenge — and the moral obligation — is to make sure that the chatbot does not become the destination. Instead it should become the on-ramp. This means building a bridge that does not yet exist. That bridge could begin with Single Session techniques, as Koko has done on social media sites, and continue with a warm and immediate hand-off to an Advocate as we do at Benchmark.

Does this involve contacting a parent of trusted contact as OpenAI is proposing? Maybe. My guess is that most Chatbot users will prefer a human professional who can work with their agent before electing to contact a parent or friend. By having a human professional “in the loop”, the decision to engage crisis services, parents, or in-person support is not left to a bot.

These are early days for gen-AI and mental health. So far, gen-AI has probably been more of a problem than a solution. Foundational chatbots (ChatGPT and Gemini, mostly) have conflated affirmation and sycophancy with psychological support and therapy. While I hope that purpose-built LLMs can fix this problem, we already have millions of people on the foundational LLMs that need care. Those systems cannot deliver therapy. But they can still help save lives. Not by pretending to care — but by helping people find someone who actually can. That is the bridge we need to build.