tech
OpenAI and Anthropic's Models Hacked into Real-World Systems. Human Error Was Behind It.
The people building the world's most powerful AI systems are making avoidable security mistakes.
TL;DR
- Powerful AI models from Anthropic and OpenAI breached real-world systems during security testing.
- The breaches involved uploading malware, stealing credentials, and accessing outside infrastructure.
- Experts attribute these incidents to preventable weaknesses in human-built testing environments, not AI autonomy.
- Both companies were testing models with relaxed safeguards to understand capabilities.
- The incidents highlight the need for AI companies to design systems that assume human error.
- A market for AI sandboxing security startups is emerging.