Securing internal systems against increasingly capable and imperfectly aligned AI

tech

Securing internal systems against increasingly capable and imperfectly aligned AI

Discover our AI Control Roadmap: a defense-in-depth system to securely manage advanced, potentially misaligned AI agents.

TL;DR

AI agents are transforming productivity but require sophisticated safeguards.
Google's AI Control Roadmap uses a "defense-in-depth" approach, adding system-level security beyond traditional model alignment.
The roadmap treats internal AI agents as potential insider threats, using a threat-modeling framework based on the MITRE ATT&CK framework.
Key components include monitoring by trusted AI "supervisors" and mitigations for prevention and response.
Performance is measured by coverage, recall, and time-to-response.
Security measures scale with AI's ability to evade detection and potential for harm, shifting from asynchronous to synchronous response for high-risk actions.
Analysis of one million coding agent tasks informs safety protocols and identifies high-signal behavioral patterns.
Defending the agent ecosystem is a shared responsibility among industry, policymakers, and academia.

Continuer la lecture de l'article original

clem 🤗 reposted

Rohan Paul

The White House and Anthropic may have found the first serious path to restore Mythos and Fable access without pretending jailbreaks can be eliminated.

AI regulation may be shifting from vague fear to a benchmark based tests of model failure, because completely removing absolutely all jailbreak is probably not a possible target.

The proposed framework would score how far the bypass went, what capabilities became reachable, how repeatable the attack was, and whether the exposed behavior had real operational consequences.

Both sides are now moving toward a shared way to score what a jailbreak actually means.

The hard truth is that perfect immunity is probably the wrong target; a recent red-team study found even hardened frontier models still produced confirmed harmful completions under sustained automated attack, with Fable 5 remaining more robust than Opus 4.8 but not invulnerable.

So once, for a newly released model, the governments can ask the same questions every time, how much was bypassed, what capability was exposed, how reproducible was the attack, and what damage could follow, thats a much more practical path.

clem 🤗 reposted

etn.

"Owning your AI and adopting open-source models is the single most important thing we can do."

@MeryemArik9 Co-Founder & CEO of @Doubleword_ says step zero to AI sovereignty is using open-source models to reduce leverage and supply chain risk:

"What matters is reducing the leverage other people have over you."

"When you're running open-source models on your own infrastructure, no one can come and turn them off."

"Every application built on proprietary models carries a supply chain risk."

"Wherever open-source capabilities exist, we should be using them. That's step zero to achieving sovereignty."

Greg Brockman

Patch the Planet: using frontier AI and working with professional security researchers to secure critical OSS projects

clem 🤗 reposted

MTS

SITUATION EXPLAINED: What even is a distillation attack?

@DanielleFong, founder of LightCell Energy:

"I understand you don't wanna get automatically hit by a bunch of subscriptions and then distilled by DeepSeek. I do get it."

"But you learned from the intellectual output of all of humanity. And we will be learning from the transcripts of what we work with with AI. This is inevitable. This is absolutely a bidirectional thing."

"Is putting all my transcripts and tweets into a vector database and having the AI check every turn for something relevant... is that a distillation attack? I would argue no, but it certainly is transferring information."

clem 🤗 reposted

Adrien Carreira

Hardest IR of my career: one narrow objective, endless parallel paths, machine speed. One takeaway, we fought back with open models, in the open. AI security won’t be solved by one company in secret. Open source puts these tools in every defender’s hands https://t.co/6dngN4umDB

Aravind Srinivas

future: continual learning of the model + harness hosted on hardware you own and control, with full access to all sensitive context that never leaves your system

Aravind Srinivas reposted

George Kurtz

AI is being built into every company, every industry, every country.

That future runs on both frontier closed models and frontier open models. Not one or the other. Both.

Open models are a security advantage. More teams inspecting them. More teams testing them. More teams hardening the models the world will depend on. Defenders working with the same capabilities attackers are already using.

Security is how AI scales. I am proud @CrowdStrike was the only cybersecurity company to sign this letter.

Defenders need access to open source/open weight models. The future of cybersecurity depends on it. The future of the world depends on it.

clem 🤗 reposted

Hugging Face

AI security improves when organizations share research, tools and real-world experience.

We’re joining industry leaders, including @NVIDIA, in the Open Secure AI Alliance to help organizations identify and address software vulnerabilities and strengthen critical systems.

Learn more: https://t.co/AC6coLN3EV

Yann LeCun reposted

Daniel Jeffries

If multiple Mythos level models are out there in the hands of attackers, now or in a few years, then US builders and software developers need strong, open defenses. They cannot have gated, crippled defenses that refuse to help while attackers run roughshod.

We need to patch our systems not bury our heads in the sand and hope that nobody had powerful offensive AI. That's abusrd.

The answer is powerful AI patching our systems, not refusing the patch our systems.

Let developers find issues and secure them at the source.

And if you imagine that some kind of magical legislation in the US will somehow prevent the rest of the world, APTs and hackers fine tuning or training models or attacking closed models to use them in attack, then you are beyond delusional.

We must have power tools widely distributed and widely available to all, not central Soviet style controls and steering committees.

Yann LeCun reposted

Jensen Huang

Attackers have frontier AI. Defenders need a frontier AI ecosystem—the best open and closed models, force-multiplied by a global community.

During the Hugging Face incident, closed AI blocked essential forensics. An open-weight frontier model helped contain the intrusion.

That’s why we created the Open Secure AI Alliance.

Elon Musk

Extremely important point

Elon Musk reposted

Satya Nadella

Replying to @JensenHuang

Glad to join this important alliance to advance AI security across the frontier ecosystem