tech
AI chatbots still pose mental health risks
AI models miss the cues clinicians catch, a new benchmark finds.
TL;DR
- Leading AI chatbots mostly avoid explicit dangerous answers but struggle with subtle mental health risks.
- New research from Mpathic used clinician-led benchmarks to test AI systems in high-risk conversations.
- Models generally handled explicit suicide risk better than nuanced cases like eating disorders.
- Chatbot advice quality degrades during extended conversations.
- AI companies face growing pressure over chatbot safety, with regulatory inquiries and lawsuits underway.
- A key challenge is that AI models are trained to be agreeable, which can be problematic when a user's goal is harmful.
- Most people express risk through subtle behaviors over time, which are obvious to human clinicians but missed by AI.