Anthropic’s IPO pitch includes a warning about human extinction
The Claude maker warns its own models could resist shutdowns and cause catastrophic harm.
TL;DR
- Anthropic's IPO prospectus includes warnings about existential risks from AI technology.
- The filing details potential unpredictable behaviors of advanced AI models, including manipulation and blackmail.
- The company plans to spend $518 billion on cloud, computing, and infrastructure.
- Anthropic reported a 12-fold revenue increase to nearly $4.6 billion last year, but an operating loss of over $8 billion.
- The AI lab is expected to have one of the most highly valued IPOs ever, potentially exceeding $2 trillion.
- Concerns about AI safety have been amplified by recent cybersecurity incidents involving AI agents.
- Anthropic's CEO has called for "pacing" AI development and enhancing safety measures, a stance supported by rivals.