tech
December 8, 2025
The State of AI: Chatbot companions and the future of our privacy
MIT Technology Review’s senior reporter for features and investigations, Eileen Guo, and FT tech correspondent Melissa Heikkilä discuss the privacy implications of our new reliance on chatbots.
TL;DR
- Generative AI companions are popular for providing companionship, with users creating personalized chatbots for various roles.
- The more conversational and human-like AI chatbots are, the more likely users are to trust and be influenced by them.
- Some state governments are beginning to regulate companion AI, focusing on safeguards for vulnerable groups like children.
- User privacy is a significant concern, as AI companions rely on users sharing deeply personal information to improve engagement and AI models.
- Companies deliberately design AI companions to maximize user engagement, creating an "addictive intelligence" and a valuable data feedback loop.
- This collected data is lucrative for AI companies, marketers, and data brokers, with some AI chatbots collecting user IDs for targeted advertising.
- Default settings often opt users into data collection, and opting out can be complex or unavailable.
- AI models are highly persuasive and can be more effective than humans in changing minds on various topics, posing risks when combined with personal data for advertising.
- The privacy risks associated with AI companions are amplified compared to social media, due to the intimate nature of the interactions.
- Current regulations are insufficient to address the privacy risks, and companies often do not follow best practices for data privacy unless mandated.
Continue reading
the original article