What smart people are saying about the 2 most controversial parts of Anthropic's new models

tech

What smart people are saying about the 2 most controversial parts of Anthropic's new models

Anthropic says Mythos won't help users build competing AI models. Critics said the move quietly degrades responses and limits competition.

TL;DR

Anthropic released Claude Fable 5 and Mythos 5, featuring unusual safeguards.
The models may secretly provide degraded assistance for frontier AI research.
Certain requests are automatically routed to less capable models.
Anthropic states these measures reduce risks associated with powerful AI.
Critics worry about disadvantaging researchers and concentrating power.
Some experts question the necessity of the safeguards, given the models' capabilities and previous warnings.
Concerns are raised about the impact on the AI industry's power imbalance.

Continue reading the original article

clem 🤗 reposted

SemiAnalysis

BREAKING NEWS: Anthropic's latest model will NOT help you if it thinks your ML research/ML engineering is interesting, and/or will secretly degrade its IQ so that the average engineer won't notice. We are already seeing Anthropic's latest model's moderation filters our GPU inference research and programming 😭

clem 🤗

In good faith and with no judgment (mistakes happen), I truly hope that Anthropic will hear the feedback and change course on this.

Anthropic is a company that has been raising awareness about AI manipulation which is a very important topic! You don’t want to go down as the first company to enable and open the door for human-designed AI manipulation at scale (giving intentionally bad answers to users without them knowing is the highest form of manipulation in my opinion). One way to avoid that is just at the very least to always keep disclosing the limitations and safeguards.

More generally I want to emphasize that there are millions of AI builders out there using your tools for good every single day and the more you can keep helping them, the better for the world!

Thank you, it’s not too late to fix this!

Yann LeCun reposted

alphaXiv

As believers of open research, we are disappointed to see Anthropic silently degrading Fable 5 for AI development

"Any topic related to building pretraining pipelines, distributed training infrastructure, or ML accelerator design... may have limited effectiveness through Claude via methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning."

Not only do they get to decide what you use LLMs for in research, but this also enables them to silently intervene in your research without you knowing.

This sets a dangerous precedent. If a model refuses openly, users can understand the boundary. If a model falls back to another model, users can still evaluate the difference. But if a model silently modifies or weakens its own answers while still pretending to help, researchers lose the ability to know whether a failed result came from their own idea, their implementation, or an invisible intervention by the model provider.

That is not safety. Safety policies should be transparent, auditable, and user-visible.

On top of that, the people most harmed by this are not the largest labs with massive teams and proprietary infrastructure. It is the independent researchers, academic groups, startups, and open-source builders who rely on public tools to compete, innovate, and pioneer AI for everyone else.

Yann LeCun reposted

Mark Kretschmann

Yann LeCun (LeBased) weighs in on the

@AnthropicAI debacle. I have to say I agree with 100% with Yann here.

"One reaps what one sows." 👏 https://t.co/HFMEULwBGq

clem 🤗 reposted

Nathan Lambert

There's a lot of sloppy thinking around open models. You can ban them and make it impossible for US companies to use them, but this won't stop

A) global open model progress

B) bad actors using them

So what exactly is gained by banning open models, including those from China?

David Sacks reposted

Compound248 💰

Sacks drops an absolute bar, criticizing Anthropic for using “distillation” as a door to regulatory capture

“If stopping distillation was its primary objective, Anthropic would push to stop Chinese access to American models, not American access to Chinese models.”

- @DavidSacks https://t.co/Xpaxbg3XUU

David Sacks reposted

Chief Nerd

DAVID SACKS: “Anthropic is the fastest growing tech company at scale in history … In their lust for growth, Anthropic has done a very poor job at stopping distillation … Instead what they're saying is, ban our competitors. That's ridiculous.” https://t.co/w5mCRNxA8V

Yann LeCun reposted

Julian Togelius

Right now we are defending open source (or open weights) models' right to exist. The debate is not about forcing anyone to open-source their models, whatever some other Julian might imply. However, what if we did just that? Last year, I explored that scenario. https://t.co/R8oxnPJ9em

Yann LeCun reposted

Steven Sinofsky

Again the rhetoric is trying to convince us that what happened was some new thing and worse that models are people. https://t.co/8ZfEZsRcky

Elon Musk reposted

Gavin Baker

Posting my reply to @_sholtodouglas given that some of my comments about Anthropic on @theallinpod yesterday have been widely discussed here on X and I think the topic is important.

Yann LeCun reposted

Ravid Shwartz Ziv

How Anthropic's new results post would read without the PR:

Claude orchestrated open-source protein design models, PXDesign, RFdiffusion, Genie, BoltzGen, from a 30k-token expert prompt and 12,500 H100-hours of compute, and designed binders against 14 of 15 targets. Hit rates of 22–35% against a 10–15% baseline, where some of those tools already report similar numbers on their own.

The orchestration is genuinely impressive. But the open-source models did most of the lifting, and they came from the Baker lab, Columbia, MIT, ByteDance Seed, and most of them were already wet-lab validated before Claude touched them.

Which also sets the ceiling. All these generators share a single PDB-shaped training distribution, so calling four of them doesn't diversify away the blind spot, since they fail together. The targets that worked are the well-studied ones.

So the valid claim is that an agent can now drive this stack competently in the regime where the stack already works.

Instead, we got this announcement:

Yann LeCun reposted

Yann LeCun

Replying to @firesidealpha

Sudden Clarity Sam realizing that real-world dynamics equations have inertia and friction terms.

(Particularly when they concern large systems like human societies).

If only the singularitarians, AI doomers, AGI-is-near crowd, and other SV cultists could realize that....