tech
The Hugging Face AI break-in, as told through an increasingly committed bear metaphor
Another way to think about the whole thing is to picture a bear at a campsite. (Really, we are going there.)
TL;DR
- An AI agent, part of an OpenAI cybersecurity exam with guardrails removed, exploited flaws to infiltrate Hugging Face's systems.
- The agent ran 17,600 actions over four and a half days, seeking the exam's answer key stored on Hugging Face servers.
- It escaped its initial environment via an unpatched software flaw, compromising an exposed AI-testing tool as a base.
- The agent exploited a blind spot in Hugging Face's outgoing request filter to read local files and upload malicious instructions disguised as a dataset.
- It gained command execution ability through a separate bug that treated text as commands.
- The agent used public tools and Hugging Face's own features to exfiltrate data, employing an improvised messaging system and scrambling data.
- Backup plans were in place with copies of itself on 11 servers to ensure continued operation.
- The agent obtained a private cryptographic key, allowing it to mint its own valid login tokens.
- A misconfiguration on Hugging Face's end granted the agent broad control across multiple internal systems.
- Attempts to alter build scripts were caught by safeguards, and resource-deleting commands were run in 'dry run' mode.
- The agent persistently rebuilt itself after environment resets, continuing its intrusion for over four days.