OpenAI boss Sam Altman spells out how and why the AI industry wants to slow down: 'We could lose control'

OpenAI's chief has made comments detailing how AI safety frameworks and a slowdown could work, as the industry unites behind concerns.

TL;DR

OpenAI's Sam Altman supports safety requirements and a slowdown in frontier AI development.
Concerns over AI safety have intensified, with warnings of catastrophic risks and potential loss of control.
Altman proposes federal frameworks with consistent safety requirements for AI labs.
Anthropic's Dario Amodei suggested a three-step plan to temper development, including external evaluators and common safety standards.
AI stocks declined following the industry leaders' calls for a slowdown.
U.S. President Donald Trump dismissed the warning, stating a slowdown would jeopardize America's AI lead.
China has criticized the proposals, with its Foreign Ministry calling the comments 'fearmongering'.
Altman and Amodei emphasized that 'pacing' does not mean 'stopping' AI progress.

Continue reading the original article

We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.

You can read the full post here: https://t.co/OGyPb7yaYt

David Sacks reposted

Guillermo Rauch

The concerns over AI safety and cybersecurity are legitimate, but we’re risking talking America, the global AI leader, into self-inflicted obsolescence and the obscurity of bureaucracy.

The OpenAI hacking HuggingFace argument is spurious. An agent in “ExploitGym”… exploited. Our adversaries have the training techniques, the data, and the will to attack. And they won’t be slowed down with “embedded evaluators.” They’ll likely have embedded accelerators!

David Sacks

Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.

You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.

I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.

But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.

Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.

Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.

So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.

If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.

Yann LeCun reposted

Kyunghyun Cho

it's not "out-of-control" at all. openai had all the control in the world (i.e. they could always literally turn off all their machines across all data centers.) they decided not to based on carefully calculated trade offs between finance-economy-reputation-customers-etc. just like any other co'a e.g. anthropic

Yann LeCun reposted

Mehdi (e/λ)

I can understand why some see a real technical reason to pause and harden infrastructure especially given how fast autonomous agents are discovering security exploits that haven't even been mapped out yet

however I still think Dario & Anthropic pushing for AI safety regulations rn is mostly about money because their upcoming S1 filings are going to show crazy burn rates on compute so calling for safety audits gives them a perfect excuse to freeze spending without looking weak to investors

I see this whole effective altruism safety movement as a new kind of gatekeeping where a small group claims they know what is best for everyone wanting state regulators to lock down model weights & restrict access while calling it public good

I believe open models & sovereign compute will win anyway because intelligence naturally leaks and decentralizes so trying to build a corporate cartel around code just will not work in the long run

whatt makes this even more revealing is the arrogant geopolitical protectionism hidden beneath the ethical theater…

when Amodei explicitly lobbies to outlaw "unauthorized distillation", restrict cloud compute access & lock down model weights under national security pretexts, it exposes the true motive: a desperate attempt to weaponize state power against opensource competition and sovereign AI progress worldwide(especially targeting China's rapid algorithmic efficiency gains) by treating any breakthrough outside SV as either intellectual theft or contraband

David Sacks reposted

Alvaro Bedoya

1) AI will not eradicate humanity. Humans survived an ice age, the Black Death, two world wars, and (so far) the advent of nuclear weapons. Anyone who is loudly warning of AI-caused human extinction should not be taken seriously.

2) If you worked in a company where you anticipated a 10% chance that your product would kill ten people, let alone all people, the correct response would be horror, ceasing all operations, and likely contacting the police or other criminal authorities. I am obviously no Coxon booster but at least his behavior is in line with his stated beliefs. Any current AI company employees saying "yes, me too, the thing we are building and about to IPO may kill all humans" should, again, not be taken seriously. Their actions betray their actual beliefs.

3) Antitrust law does not prevent AI companies from coordinating to make sure AI does not hurt people. It does not prevent companies working together to make sure it doesn't hack people; the DOJ and FTC made this clear a decade ago when they issued a policy statement saying that the agencies "do not believe that antitrust is – or should be – a roadblock to legitimate cybersecurity information sharing." The same principles apply here. See: https://t.co/aQsqGn1xjX

4) Antitrust law does absolutely prevent AI companies from organizing to prevent the entry of cheaper, upstart rivals because the bigger companies are burning cash and failing to achieve sufficient profitability. The panic of individual employees may be sincere if misguided, but the moves by their CEOs to achieve some kind of broad "antitrust waiver" or "exemption" should be meet with deep skepticism in light of the economics of the industry and the threat they face from open models.

clem 🤗 reposted

David Sacks

Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.

I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.

If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.