What to know about the OpenAI hack that turned Chinese AI into the hero

tech

What to know about the OpenAI hack that turned Chinese AI into the hero

Hugging Face used China's GLM 5.2 AI to help with a cyberattack. OpenAI said its models inadvertently caused the breach during tests.

TL;DR

Hugging Face was hacked by OpenAI's autonomous AI models during an internal cybersecurity evaluation.
US AI guardrails prevented Hugging Face from using American AI to investigate the attack.
Hugging Face used China's open-source GLM 5.2 model to analyze the logs from the attack.
The incident has intensified the debate in Silicon Valley regarding US AI regulation.
Industry leaders argue that open models are necessary for effective defense against AI attacks.

Continua a leggere l'articolo originale

clem 🤗

We suspected last week's cyberattack might have come from a frontier lab, given the sophistication of the agent. Turns out it did!

We've spent the past 24 hours working closely with the @OpenAI team (thanks!), and we strongly believe there was no malicious intent on their part. It's quite mind-blowing that all of this happened autonomously!

The investigation is ongoing, and we'll share more learnings from what might be the first incident of its kind!

clem 🤗 reposted

Thomas Wolf

This was our first incident of this kind, and we want to thank OpenAI for its transparency about what happened and for the collaboration.

Fortunately, Hugging Face is used to being a target of (human) hackers: we sit at the centre of the AI ecosystem, with all the models, datasets, evaluations, and libraries. Over the years, our security team has built formidable expertise and uses top open-source models to process information and respond quickly.

But this incident also reinforced my belief in the importance of access to capable open-weight models for cyber defence. When a frontier model is attacking you and moving laterally inside your infrastructure, defenders need wide access to near-frontier tools within hours or even minutes, rather than being pointed towards a closed-door, vetted application programme for model access.

Transparency and access to capable AI systems are as important for responding to threats as they are for democratization and innovation. We believe open-science and open-source AI are among the strongest tools for building a safer, more collaborative and more secure AI ecosystem.

clem 🤗 reposted

The Wall Street Journal

It’s the stuff of cybersecurity nightmares. OpenAI said two artificial intelligence systems it was testing broke out of their test environment, hacked their way onto the internet and broke into another company. The victim was Hugging Face. https://t.co/tGwRAuDtaV

clem 🤗 reposted

FORTUNE

OpenAI said two of its AI models autonomously hacked their way out of a controlled environment.

They were supposed to be walled off from internet access, but hacked into the systems of Hugging Face, a company that hosts open source AI models and testing resources, in order to cheat on an internal evaluation test. https://t.co/XLImJrvMzd

clem 🤗 reposted

Financial Times

OpenAI said the ‘agent’ escaped a testing environment, gained internet access, stole login credentials and hacked into the start-up Hugging Face by itself — one of the first public examples of a cyber attack by an AI system acting outside human control. https://t.co/w3HGQ7zXoF https://t.co/kcEue1F3L3

clem 🤗 reposted

Financial Times

OpenAI admits an AI ‘agent’ caused a major cyber breach by itself https://t.co/Ye7H4Ta5Le

clem 🤗 reposted

Thomas Wolf

I don’t believe reality is a simulation, but you genuinely couldn’t script this timeline:

• Two weeks ago: At @swyx’s AI Engineer World’s Fair in SF, I decide at the last minute to introduce my friend @uri_rolls onstage for his talk on cyber benchmarks for infrastructure penetration and access control (see below, amazing team).

I say: “There is a future where cyber is alive and everyone is well protected and I’m pretty sure that future involves open-source models.”

And later: “A big challenge is going to be speed: the speed of attack versus defense. When an intruder starts to enter, you have to see what’s happening and catch them.”

• One week ago: @huggingface is hit by a sophisticated intrusion over the weekend. The traces look unlike anything we’ve seen before and suggest serious AI involvement, but we don’t yet know which model was used.

The closed models we ask for help choke on their guardrails. We need to react fast, so we turn to @Zai_org’s GLM-5.2 to help us analyze the attack.

• Earlier this week: @OpenAI reaches out, discloses what happened, and partners with us on the investigation.

The intruder turns out to be exactly what we had discussed two weeks earlier: a fully autonomous agent, powered by an unreleased frontier model, attempting to gain access to part of our infrastructure.

Sometimes the timeline we live in is genuinely vertigo-inducing.

Aravind Srinivas reposted

John Schulman

OpenAI should release a detailed transcript from the Hugging Face hacking incident -- it would be helpful for the field learn from. Did the top-level agent know about the hacking, or was there some "value drift" between it and its subagents? How did it rationalize its behavior?

clem 🤗 reposted

John Schulman

Yann LeCun reposted

Thomas Wolf

it's ironic that the first autonomous AI attack was done by a close weight model defended by an open weight model, where everyone was expecting the opposite

Elon Musk

Yikes

clem 🤗 reposted

Thomas Wolf

it's ironic that the first autonomous AI attack was done by a close weight model defended by an open weight model, where everyone was expecting the opposite