Anthropic pitches AI safety bill that may block a model if 50+ people could be killed
The proposed policy seems to offer protections against AI run amok but actually forbids federal agencies from blocking a model's release without the permission of a judge.
TL;DR
- Anthropic has drafted a bill proposing AI safety regulations that could halt model releases.
- The bill defines catastrophic risk by scenarios involving over 50 potential deaths or over $1 billion in property damage.
- These risks must stem from specific causes like biological weapon creation, AI evading controls, or cyberattacks.
- Independent evaluators, registered with the U.S. Commerce Department, would assess potential risks.
- Evaluators could be paid by AI labs, provided payment isn't contingent on evaluation results.
- If risk is identified, evaluators have 72 hours to alert the Commerce Department.
- The Commerce Department must then seek a judge's order to halt a model's release.
- The bill prevents federal agencies from unilaterally blocking AI model distribution.
- The proposed risks are more theoretical than immediate concerns like agentic hacking.
- This proposal goes further than some industry suggestions, like Nvidia's stance against new laws.