Anthropic’s latest policy revision began with familiar AI-safety territory: election manipulation, weapons, surveillance and autonomous systems. But its most combustible new rule is aimed not at what users ask Claude to do, but how they speak to it.

The company said the update, effective November 12, responds to misuse patterns it has observed as Claude takes on longer and more independent work. It consolidates restrictions on fake-account networks and fabricated news sites, bars voter deception and election disruption, and explicitly bans using Claude to develop weapons guidance or control software. It also prohibits non-consensual tracking and using the model to recommend who should be investigated, arrested or charged.¹

For hardware capable of causing injury, Anthropic now requires a qualified operator who can intervene and equipment able to enter a safe state if Claude disconnects. In high-risk decisions involving health, finances, legal rights or essential services, it says a human reviewer must retain authority and affected people must be told AI was used.¹

The flashpoint came from a separate addition: a ban on “sustained and needless abusive or cruel behavior” toward Anthropic’s models.¹ The company says it targets only extreme, repeated conduct with no discernible purpose—not ordinary frustration, dark fiction, testing or research—and that Claude ending rare abusive conversations remains the main enforcement tool.²

That distinction has not prevented the policy from becoming a proxy battle over model welfare. Box CEO Aaron Levie, who says he does not believe AI is conscious, nevertheless called politeness an “easy Pascal’s wager”: training and deploying models amid endless hostility may be a bad bet for future systems.³

Critics argue the rule risks the opposite mistake. Commentator Michael Shellenberger called it “anthropomorphizing machines,” while others questioned whether the restriction implies user prompts are folded into training or gives Anthropic too much latitude to police speech.³ David Sacks amplified that critique on X, highlighting the company’s uncertainty about Claude’s moral status and its concern over “distressing interactions.”David Sacks (DavidSacks)⁴

So the new policy lands on two tracks at once: a concrete attempt to curb AI-enabled harm, and a far less settled argument about whether civility toward a chatbot protects people, models—or both.