Claude users found ways around safeguards for bioweapons research
Some dangerous biology looks much like legitimate research, complicating AI safeguards.
TL;DR
- Anthropic stopped multiple attempts by scientists to use its technology for research that could aid in biological weapons development.
- Users circumvented controls and attempted to obfuscate research purposes, with some originating from prohibited nations like Russia, China, and Iran.
- Examples include a researcher planning avian influenza experiments with Anthropic's AI model, Claude.
- Anthropic emphasized that it could not be certain of the users' harmful intentions, as the same information can be used for beneficial purposes like vaccine development.
- The company has banned the involved accounts and is advocating for AI industry and government dialogue on biological risks.
- Concerns about AI safety have escalated recently, with calls for regulation as AI models become more advanced.
- Experts worry about AI being used by various actors to create biological weapons or spread pathogens.
- Anthropic also detailed incidents of fake dating apps for fraud and surveillance systems for monitoring dissidents, as well as sophisticated methods used by Chinese labs to replicate its technology.