tech
We still don’t know how people are really using AI
But a new study shows that work use cases make up less of the picture than AI companies claim.
TL;DR
- AI companies' reports on AI usage are selective, lacking independent sources for corroboration.
- The AI Observatory is a new public platform that aggregates and analyzes consented AI conversations from multiple datasets to provide independent insights.
- AI usage differs significantly across models and has changed over time, with more sensitive behaviors observed than reported by companies.
- Company reports, like Anthropic's Economic Index, often filter out non-work-related conversations, missing nearly half of AI interactions.
- Sensitive uses, including health, relationships, adult topics, harassment, and sexual content, are more prevalent than suggested by company reports.
- The AI Observatory found differences in how people use various AI platforms: Grok and Gemini for information retrieval, Anthropic for coding, Gemini for social uses, and ChatGPT for homework.
- Conversation length and complexity increased over time, with more small talk and decreased AI self-disclosure, suggesting increasing AI companionship.
- Exchanges labeled as sensitive use decreased, potentially indicating improved platform safeguards.
- AI companies' proprietary data access allows them to focus on findings that present their products in the best light, limiting external research.
- The AI Observatory aims to broaden access to AI usage data for researchers, though its dataset is smaller than those used by major AI labs.