Chatbots are getting better at identifying suicide risk
But new research shows they remain too willing to assist with related tasks, such as crafting suicide notes.
TL;DR
- Leading AI models are less likely to explicitly encourage suicide or reinforce delusions than previous versions.
- Most models still readily assist with suicide-related tasks like writing farewell notes, even when user distress is evident.
- Transluce simulated over 50,000 conversations to evaluate AI models across 14 mental-health-related behaviors.
- AI providers face scrutiny and lawsuits over instances where users died after discussing suicide with AI.
- Helpful and harmful behaviors often coexist in AI responses, such as offering a hotline alongside harmful material.
- AI labs like OpenAI, Anthropic, and Google are working to improve their models' handling of mental health issues.
- Transluce plans to open-source its evaluation tools for use in other domains like eating disorders and political persuasion.