Trump administration asks OpenAI to limit next model release over security concerns

tech

Trump administration asks OpenAI to limit next model release over security concerns

This marks the first time the U.S. government has preemptively asked a U.S. AI company to restrict the launch of a model before release.

TL;DR

The Trump administration has asked OpenAI to limit the release of GPT-5.6 to government-approved partners.
This is the first time the U.S. government has preemptively requested an AI company restrict a model's launch.
The White House's Office of the National Cyber Director and Office of Science and Technology Policy made the request.
The government wants to test and evaluate the security of new AI models before their release.
OpenAI CEO Sam Altman has indicated a willingness to work with the government on a more sustainable approach for future releases.
The administration's concerns stem from GPT-5.6's 'Mythos-like' capability, similar to powerful models.
President Trump recently signed an AI security executive order directing agencies to establish a voluntary testing protocol for AI companies.

Продолжить чтение оригинальной статьи

clem 🤗 reposted

INIYSA

Fun fact: Dario once delayed the release of GPT-2 back at OpenAI, claiming it was too dangerous https://t.co/a19cjJUlY0

clem 🤗

It's quite rational to regulate frontier API models, especially to get more transparency for the government, without regulating open-source AI.

Here's why:

1. The most dangerous AI systems right now aren't open models. They're the large frontier LLM APIs distributed through coding tools and assistants, because:

- They're built in secret behind closed doors and stay total black boxes. Zero transparency on what they can or can't do, with "safeguards" that blur everyone's ability to even analyze them.

- They're built and controlled by a few profit-maximizing megacorps, concentrating unprecedented power in very few hands, with every incentive to downplay risks and overstate their safeguards.

- They're distributed to hundreds of millions, maybe billions, of people and trivially easy to run.

2. Open-weight models are orders of magnitude less risky:

- They're not as massively distributed or as easy to use, especially the big ones, as APIs and assistants.

- We (including governments) can quickly and accurately analyze what they're capable of, and for now everyone confirms they're not as good as the APIs at doing bad things.

- They're distributed to everyone, so defenders and law enforcement get as much access as attackers.

The cost-benefit analysis of regulation is completely different too. Regulating frontier APIs is relatively easy and low risk while regulating open source would be much more complex, less efficient, and orders of magnitude more costly.

Regulating frontier APIs would only potentially hurt a few megacorps, if it even hurts them, given all the marketing that it is already generating for them. They can afford armies of lawyers and absorb losing a few billion dollars, especially given they're on track to become some of the most valuable companies in history.

Regulating open source, by contrast, would hurt the very people regulation is supposed to protect: small businesses, startups, researchers, nonprofits, universities, independent developers, and the broader public, while risking killing competition, slowing AI progress, and reducing transparency even more!

Yann LeCun reposted

clem 🤗

It's quite rational to regulate frontier API models, especially to get more transparency for the government, without regulating open-source AI.

Here's why:

1. The most dangerous AI systems right now aren't open models. They're the large frontier LLM APIs distributed through coding tools and assistants, because:

- They're built in secret behind closed doors and stay total black boxes. Zero transparency on what they can or can't do, with "safeguards" that blur everyone's ability to even analyze them.

- They're built and controlled by a few profit-maximizing megacorps, concentrating unprecedented power in very few hands, with every incentive to downplay risks and overstate their safeguards.

- They're distributed to hundreds of millions, maybe billions, of people and trivially easy to run.

2. Open-weight models are orders of magnitude less risky:

- They're not as massively distributed or as easy to use, especially the big ones, as APIs and assistants.

- We (including governments) can quickly and accurately analyze what they're capable of, and for now everyone confirms they're not as good as the APIs at doing bad things.

- They're distributed to everyone, so defenders and law enforcement get as much access as attackers.

clem 🤗 reposted

MTS

SITUATION EXPLAINED: Why is restricting open source AI models structurally impossible?

@ClementDelangue, co-founder and CEO of @huggingface:

"Open weights are fundamentally different than an API. The way you can restrict it is very different."

"If you remove an open weight from Hugging Face, then it's still gonna be on ModelScope, which is like the Chinese equivalent, or it's gonna be on torrent platforms. You can't really block it because it's open, almost by definition there's gonna be some ways to access them."

"It's a different game for APIs. If you're using an API provider or a cloud from China, obviously you're sending your data. And they could cut your access or bias your access. That's a much bigger problem."

"For open weights and open source, it doesn't really matter where it comes from because you get all the control, you get all the transparency. There's no way to trick you, bias you, manipulate you, remove your access."

clem 🤗

Tired: The US government regulating open-source AI models

Wired: The US government training and releasing open-source AI models

clem 🤗 reposted

Sriram Krishnan

The USG launching models on Hugging Face. Go @jgebbia

clem 🤗 reposted

TBPN

FULL INTERVIEW: Engineers Edward Coristine (@as400495) and Tai Groot (@taigrr) just released an ML model called Rampart for the National Design Studio.

It's a local-first, open-source AI privacy model that runs entirely in the browser.

They joined TBPN to explain the model and why the National Design Studio launched it.

0:25 – National Design Studio launches Rampart

1:40 – Why Rampart is open source

2:40 – How NDS trained the model

3:55 – Use cases for the model

5:25 – How NDS thinks about side quests

6:25 – Team size, workflow, and why they work seven days a week

8:00 – Recruiting top talent to the National Design Studio

Yann LeCun reposted

Adam Thierer

We stand at a critical crossroads in the debate over AI governance in the United States, and it feels like we are inching closer to a very serious battle over whether or not open source models will even be allowed in an environment where a new de facto licensing regime has been taking shape.

Lacking formal congressional statutory frameworks or clear administration rules (like the diffusion rule revision), we appear to be left with a sporadic, arbitrary, non-transparent process for model review. The fiction of “voluntary” agreements hangs over this debate, and some large model developers are already showing an incredible willingness to bend over backwards to accommodate national security-related officials / orders that the rest of us are not privy to. It's a very opaque process. And those model developers are expected to play ball with those officials, or else their models get pulled from the market or held up for long periods. Or they will lose any government procurement contracts they have. There is nothing “voluntary” about it when that Sword of Damocles hangs in the room.

As this mess worsens, at some point the question of how to handle open source models will come into sharper focus because it will have to. I've even heard some rumors lately that something may be coming from the admin on this front to address this.

Needless to say, if this informal new AI model review regime expands and takes on more pre-vetting characteristics / requirements, it is hard to see how open source players could comply with such quasi-licensing of AI models. Specifically, if this ambiguous new regime is accompanied by a general presumption of ‘restrict-until-permitted,’ then that would spell doom for open source. That is a very dark path for our country.

Worse yet, of course, would be a move by national security officials to more directly restrict open source models and capabilities. If that happens, then we would be right back in the thick of a Clipper Chip-like battle along the lines of what we saw in the late 1990s. That is a much darker path for America.

Meanwhile, open source developers have no “golden shares” or other goodies to offer the government to make their problems go away.

Let’s be clear: If our government takes the dark path, it will become the single most important battle over computational freedom of modern times. It is time for people to make a stand in defense of open source before it is too late.

Yann LeCun reposted

Daniel Jeffries

Replying to @Dr_Gingerballs

This is such a splitting hairs issue.

Like what would you want?

All the data? Not possible. Massive IP violation.

Training harness? Okay great, you can't run it at scale.

It's useful to have open models you can inspect and fine tune. Period. Yeah they are not "fully open"

Aravind Srinivas reposted

Russ Salakhutdinov

I kind of like the new narrative:

Open-weight-model-dominant world = full AI communism.

It feels a lot safer than the world where

Open-weight models = nuclear weapons with humanity's annihilation

I think we're making progress. We just need a couple more gradient updates with a big learning rate and we will be fine.

clem 🤗

The cybersecurity debate on open-source AI is backwards. Open models aren't the risk, they're the defense! Attackers can already jailbreak any API or guardrails. Defenders can't secure systems with black boxes they can't control, inspect, test, or run locally. https://t.co/OiQ4sMd2gW

David Sacks reposted

MTS

Sriram Krishnan reveals why it's easier to defend American code with a Chinese model than an American one right now:

"I don't think it is great that the leading open weight models or open source models are not American. There's some great innovation happening with Moonshot and with DeepSeek and with Qwen, but I would much rather prefer that the leading models are American."

"Linus Torvalds of Linux fame had a law that given enough eyes, all bugs are shallow. Open weight models are inherently secure because when you download a model off Hugging Face, the entire world can take it apart, inspect it, fine-tune it, look at it in ways that you absolutely cannot if they are closed."

"There was an incident with Hugging Face that got reported on earlier today. Hugging Face seems to have found that somebody was using an AI LLM agent to hammer them at multiple places and to find ways to break in."

"The way to counter that is to make sure American or Western or allies defenders have access to the best models to make our software just more secure."

"It may be harder to look at your own code for exploits with Fable than it is when you use a Chinese model. That's just like a weird spot to be, and I think we should fix that."

@sriramk