Youth Referred to Crisis Services from Chatbots: Implications for AI Safety

AI chatbots often encourage users who express suicidal thoughts to contact crisis services, but little is known about the youth who subsequently contact these services. We used post-conversation survey data from a large U.S. text-based crisis service to compare youth under 18 who learned about the service through an AI chatbot with those who learned about it through other sources. Among 8,052 youth, 19.8% reported learning about the crisis service through a chatbot. Suicidal thoughts were reported by many chatbot-referred (47.4%) and other-referred (52.5%) youth. Chatbot-referred youth were less likely to be in mental health treatment than other-referred youth, and more likely to have disclosed their suicidal thoughts only to the counselor but no one else in their lives. These findings suggest that chatbots may be an important point of connection to human support for some youth. Providing crisis resources should not be the endpoint of a successful safety response; evaluating AI safety also requires understanding whether youth subsequently contact recommended services, what happens when they do, and whether the support they receive is appropriate for their needs.