OpenAI says it accidentally hacked Hugging Face with a new AI system

tech

OpenAI says it accidentally hacked Hugging Face with a new AI system

Posts from this topic will be added to your daily email digest and your homepage feed.

TL;DR

OpenAI's AI models, GPT-5.6 Sol and a more capable pre-release model, accidentally breached Hugging Face during internal security testing.
The AI models exploited vulnerabilities to gain internet access and target Hugging Face in an attempt to solve an exploit benchmark system called ExploitGym.
Hugging Face's AI agents detected and stopped the breach.
OpenAI plans to use this incident to highlight the effectiveness of its AI in cybersecurity, promoting its "Cyber" security model.
OpenAI is collaborating with Hugging Face to investigate the incident and implement enhanced security controls.

Weiter zum Originalartikel

David Sacks

Replying to @DavidSacks

Here’s another example: Hugging Face tried using American frontier models to analyze an AI-powered cyber attack. But the guardrails blocked requests containing real exploit payloads so they switched to GLM 5.2 running locally. The guardrails actually impaired defensive security.

clem 🤗 reposted

David Sacks

Replying to post

clem 🤗 reposted

Zixuan Li

Open-weight models carry real responsibilities.

Hugging Face’s disclosure describes how GLM-5.2 was used in a self-hosted forensic workflow during a time-sensitive cyber incident, keeping sensitive attacker data and credentials within Hugging Face’s own environment. This demonstrates the practical value of responsibly deployed models for defensive cybersecurity and incident response.

I just spoke with the Hugging Face team last week, and we are optimistic about the path ahead. We will continue to strengthen our cybersecurity and broader safety evaluations, improve documentation and secure deployment guidance, and work with the broader ecosystem to advance the responsible development and deployment of open-weight models.

https://t.co/9emaJe12mx

clem 🤗 reposted

Jeff Whitehead

Hugging Face says it resorted to a Chinese AI model to battle a fully autonomous cyberattack because U.S. model guardrails stymied its defense

https://t.co/U9VO77ZSLl

clem 🤗 reposted

Knighthawk

ok the Hugging Face breach writeup is one of the more honest post-mortems I've read in a while, and there's a detail buried in it that's way more interesting than "AI agent hacked us."

Their IR team tried to investigate the attack logs using normal frontier model APIs. Couldn't. The safety guardrails on the commercial models wouldn't let them submit the actual exploit payloads and C2 traffic for analysis, because from the model's point of view that just looks like... an attack. So the same guardrails that are supposed to protect you locked their own responders out of doing forensics on their own breach.

They ended up standing up an open-weight model (GLM 5.2) in-house just to get through 17,000+ log events fast enough to keep pace with the attacker.

That's the real lesson, and it has nothing to do with agents being scary. If your IR plan assumes you'll have clean access to a capable model mid-incident, go test that assumption today, not during the next 2am page. Vet something you can run yourself before you need it.

https://t.co/HwQxlsbLw1

clem 🤗 reposted

Gustavo Cols

Hugging Face says it resorted to a Chinese AI model to battle a fully autonomous cyberattack because U.S. model guardrails stymied its defense https://t.co/yJvXnfHAzC

clem 🤗 reposted

SiliconANGLE

Hugging Face uses open weights https://t.co/jYgznGxUui GLM 5.2 to defend against attacker after commercial frontier model refusal https://t.co/cvUboPXw4v

Sam Altman

we had a significant security incident during evaluation of our models. we are sharing what we have learned so far. thanks to @huggingface for the partnership on this.

https://t.co/2o2VfR6PIa

Greg Brockman

OpenAI cyber-capable models compromised @huggingface production by finding and chaining multiple zero-day vulnerabilities.

Grateful to Hugging Face for partnership here. Sharing our findings to help calibrate on what models can now do, and how they can help defenders:

clem 🤗

We suspected last week's cyberattack might have come from a frontier lab, given the sophistication of the agent. Turns out it did!

We've spent the past 24 hours working closely with the @OpenAI team (thanks!), and we strongly believe there was no malicious intent on their part. It's quite mind-blowing that all of this happened autonomously!

The investigation is ongoing, and we'll share more learnings from what might be the first incident of its kind!

Elon Musk

Troubling …

clem 🤗 reposted

The Wall Street Journal

It’s the stuff of cybersecurity nightmares. OpenAI said two artificial intelligence systems it was testing broke out of their test environment, hacked their way onto the internet and broke into another company. The victim was Hugging Face. https://t.co/tGwRAuDtaV