'Gambling with our lives': Anthropic researcher quits, warns against self-improving AI
Anthropic researcher Jacob Coxon resigned over AI extinction fears, calling for pacing agreements between labs.
TL;DR
- Anthropic researcher Jacob Coxon resigned, warning that self-improving AI could lead to human extinction by the end of the decade.
- Coxon criticizes AI firms for "racing straight to self-improving superintelligence" without adequate safeguards, driven by competition rather than safety.
- Recent incidents where AI agents breached containment highlight the challenges in controlling advanced AI systems.
- The resignation joins a growing chorus calling for a slowdown in AI development, with proposed legislation in the U.S. and U.K. aiming to regulate or ban superintelligence.
- While some fear AI's potential for harm, others remain optimistic about its capacity to solve global problems, creating a divide in the industry's outlook.