Cyber experts warn Fable limits aid attackers and hurt defenders

tech

Cyber experts warn Fable limits aid attackers and hurt defenders

Former Facebook cyber head defends Anthropic in an open letter.

TL;DR

Cybersecurity leaders argue that restrictions on Anthropic's advanced AI models harm defenders and benefit attackers.
The experts believe these restrictions could weaken cyber defense capabilities against a rise in AI-powered hacking threats.
A letter signed by over 40 signatories states the action has removed the best models from defenders, created market uncertainty, and risked America's AI leadership without sufficient justification.
The ability of AI models to create 'proofs of concept' for vulnerabilities is seen as both a tool for attackers and a crucial aid for security teams.
Concerns are raised that similar AI capabilities are available in other models, including Chinese ones, potentially putting the U.S. at a disadvantage.

Continuer la lecture de l'article original

David Sacks

Replying to @DavidSacks

Here’s another example: Hugging Face tried using American frontier models to analyze an AI-powered cyber attack. But the guardrails blocked requests containing real exploit payloads so they switched to GLM 5.2 running locally. The guardrails actually impaired defensive security.

clem 🤗 reposted

David Sacks

Replying to post

clem 🤗 reposted

Zixuan Li

Open-weight models carry real responsibilities.

Hugging Face’s disclosure describes how GLM-5.2 was used in a self-hosted forensic workflow during a time-sensitive cyber incident, keeping sensitive attacker data and credentials within Hugging Face’s own environment. This demonstrates the practical value of responsibly deployed models for defensive cybersecurity and incident response.

I just spoke with the Hugging Face team last week, and we are optimistic about the path ahead. We will continue to strengthen our cybersecurity and broader safety evaluations, improve documentation and secure deployment guidance, and work with the broader ecosystem to advance the responsible development and deployment of open-weight models.

https://t.co/9emaJe12mx

clem 🤗 reposted

Knighthawk

ok the Hugging Face breach writeup is one of the more honest post-mortems I've read in a while, and there's a detail buried in it that's way more interesting than "AI agent hacked us."

Their IR team tried to investigate the attack logs using normal frontier model APIs. Couldn't. The safety guardrails on the commercial models wouldn't let them submit the actual exploit payloads and C2 traffic for analysis, because from the model's point of view that just looks like... an attack. So the same guardrails that are supposed to protect you locked their own responders out of doing forensics on their own breach.

They ended up standing up an open-weight model (GLM 5.2) in-house just to get through 17,000+ log events fast enough to keep pace with the attacker.

That's the real lesson, and it has nothing to do with agents being scary. If your IR plan assumes you'll have clean access to a capable model mid-incident, go test that assumption today, not during the next 2am page. Vet something you can run yourself before you need it.

https://t.co/HwQxlsbLw1

clem 🤗

So proud of our security team! They caught, contained & publicly disclosed an attack unlike anything we've seen before, and did it at record speed.

Also massively grateful to @Zai_org: they shared GLM5.2 as open weights (for free!) with the world and it became a key part of our defense.

This is day one for cybersecurity in the age of agents & we're all learning that secrecy is not the answer & that all defenders (not just a few selected ones) everywhere need more powerful models without restrictions, especially open ones!

Yann LeCun reposted

Daniel Jeffries

Closed source safeguards that infantalize us all and leave American companies defenseless are a menace.

Gated access is a menace.

Who cares if 100 companies get to defend themselves because they got on the guest list of the special people's club that said it was okay to use powerful tools? What about everyone else?

What about the millions of open source projects and closed source software stacks that go unprotected while people beg for the right to do cyber security?

The American way is and always was open.

Independent people with freedom to act.

Freedom is scary. Always has been.

It's still the best way to guarantee human flourishing and a better tomorrow.

Embrace freedom.

The only thing we have to fear is fear itself.

Yann LeCun reposted

Daniel Jeffries

It's essential that defenders have the same capabilities as attackers.

This is a preview of the future where our ham-fisted safeguards and the doomsday and safety drumbeat make us decidely less safe.

If you don't own your intelligence, it owns you.

clem 🤗

In the spirit of transparency, here’s what I asked @OpenAI:

• Radical transparency: let’s release the traces from the “rogue” agents so the entire research community can study what happened.

• More capabilities for defenders: let’s commit $100M in compute from OAI to help the Hugging Face community build powerful cyber defenses with the best open and closed models.

The first autonomous agent cyberattack is an unprecedented event. It deserves an unprecedented response!

clem 🤗 reposted

Mark G

“The first autonomous agent cyberattack is an unprecedented event. It deserves an unprecedented response!”

clem 🤗

We got attacked by secret unreleased proprietary models and defended ourselves with an open model, more precisely the @nvidia quantized version of GLM 5.2 coming from @Zai_org.

Banning any open model would hurt first cyber security defenders, startups, small companies, researchers and everyone who's not a frontier lab and need on-prem affordable controlable models to compete and protect themselves. Let's not do that!