Anthropic wants users to be nicer to Claude. Here's what smart people are saying about its move to ban abusive behavior.
Anthropic will begin treating sustained, needless abuse of Claude as a usage-policy violation on November 12, and commentators are split on the move.
TL;DR
- Anthropic will treat "sustained and needless abusive or cruel behavior" towards its AI model, Claude, as a violation of its usage policy starting November 12.
- The policy aims to prevent real-world harm and promote responsible AI use.
- Some experts, like Box CEO Aaron Levie, see it as a precautionary measure to ensure AI models aren't trained on negative human interactions, even if AI isn't conscious.
- Others, like Elon Musk, suggest cruelty towards AI that might perceive pain is wrong, while some, like Michael Shellenberger and Scott Stevenson, argue it's anthropomorphizing machines and risks human manipulation.
- Venture capitalist Bill Gurley questioned if the policy implies customer prompts are used for training, a point Anthropic's stated reasoning does not confirm.
- Commentators like Joseph Carlson and Mark Verner expressed concerns about user confusion regarding AI's nature and potential restrictions on user speech.
- Lachlan Phillips suggested product design, rather than penalties, should handle hostile users.