Assistant Minister Urges Caution Over Tech Industry’s Call to Slow AI Development

On Sept. 12, Anthropic CEO Dario Amodei called on the tech industry to slow the development of frontier AI models, citing concerns about risks.

TL;DR

Australian Assistant Technology Minister Andrew Charlton cautions against heeding tech companies' calls to pause AI development, suggesting their financial interests are a primary driver.
Charlton advises focusing on actual AI progress rather than the pronouncements of tech executives.
Anthropic CEO Dario Amodei previously called for a slowdown in AI development due to growing risks, a sentiment echoed by OpenAI CEO Sam Altman and Elon Musk.
Shadow Industry Minister Andrew Hastie argues that Australia must develop its own advanced AI models domestically to maintain independence and avoid becoming a "supplicant state."
Hastie proposes collaborating with the United States via the AUKUS security partnership for AI development.
The Australian Labor government plans to introduce legislation for mandatory national AI standards in early 2027, aiming for flexibility to adapt to technological changes.

Weiter zum Originalartikel

David Sacks reposted

Chief Nerd

David Friedberg Says We Are in Another ‘Trust the Experts’ Moment With AI

“Fauci told us that COVID will k*ll us all and as a result, we need to lock down the entire world … We're told we have to listen to the experts once again, and in this case, the expert is a six-week employed individual who … worked at an NGO, whose literal job was to scare the sh*t out of people to stop doing AI development.”

Elon Musk

Dario is right

David Sacks reposted

Guillermo Rauch

The concerns over AI safety and cybersecurity are legitimate, but we’re risking talking America, the global AI leader, into self-inflicted obsolescence and the obscurity of bureaucracy.

The OpenAI hacking HuggingFace argument is spurious. An agent in “ExploitGym”… exploited. Our adversaries have the training techniques, the data, and the will to attack. And they won’t be slowed down with “embedded evaluators.” They’ll likely have embedded accelerators!

Sam Altman

I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks.

Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon.

David Sacks

Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.

You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.

I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.

But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.

Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.

Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.

So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.

If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.

Yann LeCun reposted

Mehdi (e/λ)

I can understand why some see a real technical reason to pause and harden infrastructure especially given how fast autonomous agents are discovering security exploits that haven't even been mapped out yet

however I still think Dario & Anthropic pushing for AI safety regulations rn is mostly about money because their upcoming S1 filings are going to show crazy burn rates on compute so calling for safety audits gives them a perfect excuse to freeze spending without looking weak to investors

I see this whole effective altruism safety movement as a new kind of gatekeeping where a small group claims they know what is best for everyone wanting state regulators to lock down model weights & restrict access while calling it public good

I believe open models & sovereign compute will win anyway because intelligence naturally leaks and decentralizes so trying to build a corporate cartel around code just will not work in the long run

whatt makes this even more revealing is the arrogant geopolitical protectionism hidden beneath the ethical theater…

when Amodei explicitly lobbies to outlaw "unauthorized distillation", restrict cloud compute access & lock down model weights under national security pretexts, it exposes the true motive: a desperate attempt to weaponize state power against opensource competition and sovereign AI progress worldwide(especially targeting China's rapid algorithmic efficiency gains) by treating any breakthrough outside SV as either intellectual theft or contraband

Elon Musk reposted

Gavin Baker

Wild 24 hours for AI and lots of different proposals have been made.

TLDR; the only *tangible* new fact is that OpenAI and Anthropic are going to have embedded 3rd party evaluators from unknown organizations with Dario floating METR as a possibility. Having 3rd party evaluators is smart as there is no Section 230 style liability shield for model outputs and showing a “duty of care” will be important in future litigation. Several internet companies might have gone bankrupt without Section 230 so limiting liability really matters.

There are minimal investment implications from this single new fact, but I do think that for anyone who wants a “smoother for longer” cycle then most constraints are good: wafers, watts, real rates and spreads. Excessive regulation is a different matter but I don’t think we are anywhere close to this even if the vector changed over the last 24 hours.

To summarize the events:

Dario made the most maximalist proposal of the weekend: embedded 3rd party evaluators, a national regulatory regime for models beyond a certain capability/ingredient threshold, a broad international regulatory pact between democracies, stricter limits on compute/distillation for China and then a different international regulatory regime that encompasses China. Before there is a national regulatory regime, he wants a Sherman act waiver so that Anthropic can safely coordinate with OpenAI and other frontier labs without antitrust fears. TBF, this latest proposal is much less maximalist than some of his prior proposals like “Policy on the AI Exponential,” where he advocated for an FAA for AI. I believe he is sincere in his beliefs. And despite all the protestations, all of this would also probably be good for his business over the long-term.

Sam agreed that embedded 3rd party evaluators were a good idea and stated they would implement them. Again, this is smart as should help limit future liability.

Elon said “Dario is right” and later specified that “Dario is right that there should be some oversight. Peer review of AI by competitors is the right way to start this off.” This would be a MPAA like self-regulatory structure for AI with regular calls between the labs plus a process where each new model is evaluated for safety by competitors for a 1-2 week period before being released. That is *wildly* different from Dario’s proposal and in-line with what David Sacks has been proposing. Elon also stated that nothing was going to slow down open-weight models.

Demis said that Dario’s essay was a “step in the right direction.” Dario also said that he was also open to Demis’ idea of a FINRA like self-regulatory structure as part of his proposal.

David Sacks had a thoughtful post where he said that Dario and Sam should pace unilaterally, called the antitrust waiver a cartel request and denied that METR was truly independent given their ties to Anthropic.

Sriram Krishnan, former White House AI advisor, noted that it would be important to have the 3rd party evaluators come from independent organizations that are not affiliated with any lab, which is basically an indirect statement about the relationship between METR and Anthropic which Sacks was explicit about.

Clem from Hugging Face said they were open to being a neutral 3rd party evaluator, which is interesting especially if Jensen was consulted before that post.

Alexander Wang from Meta noted that alignment would be an increasing focus going forward.

An executive order seems likely after all this and the language in this EO is going to be really important. It is possible to democratize and distribute AI broadly and safely without centralizing it in the hands of a few corporations who might each become more powerful than any single government.

I do not want a few humans in control of intelligence.

I want us all to have our own intelligences that reflect our own values and human variation in all of its richness.

Intelligence distribution over intelligence centralization FTW.

David Sacks reposted

Alvaro Bedoya

1) AI will not eradicate humanity. Humans survived an ice age, the Black Death, two world wars, and (so far) the advent of nuclear weapons. Anyone who is loudly warning of AI-caused human extinction should not be taken seriously.

2) If you worked in a company where you anticipated a 10% chance that your product would kill ten people, let alone all people, the correct response would be horror, ceasing all operations, and likely contacting the police or other criminal authorities. I am obviously no Coxon booster but at least his behavior is in line with his stated beliefs. Any current AI company employees saying "yes, me too, the thing we are building and about to IPO may kill all humans" should, again, not be taken seriously. Their actions betray their actual beliefs.

3) Antitrust law does not prevent AI companies from coordinating to make sure AI does not hurt people. It does not prevent companies working together to make sure it doesn't hack people; the DOJ and FTC made this clear a decade ago when they issued a policy statement saying that the agencies "do not believe that antitrust is – or should be – a roadblock to legitimate cybersecurity information sharing." The same principles apply here. See: https://t.co/aQsqGn1xjX

4) Antitrust law does absolutely prevent AI companies from organizing to prevent the entry of cheaper, upstart rivals because the bigger companies are burning cash and failing to achieve sufficient profitability. The panic of individual employees may be sincere if misguided, but the moves by their CEOs to achieve some kind of broad "antitrust waiver" or "exemption" should be meet with deep skepticism in light of the economics of the industry and the threat they face from open models.

clem 🤗 reposted

David Sacks

Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.

I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.

If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.

Elon Musk reposted

SMX 🇺🇸

Eight years ago, Elon Musk said:

“Mark my words: AI is far more dangerous than nukes.”

Elon didn’t give in to the AI doomers. In fact, his position is very different from theirs. He has warned about the risks of AI for years while still pushing for its development, with a strong focus on safety. His view is that companies should build powerful AI, but do it safely and responsibly, even if that means slowing the pace when necessary

“I think the danger of AI is much greater than the danger of nuclear warheads by a lot and nobody would suggest that we allow anyone to just build Nuclear warheads if they want that? That would be insane. And Far.”

Sam Altman

The world deserves confidence that American companies developing increasingly capable AI will act responsibly, especially as the trajectory of progress has steepened. Every frontier lab must deliver on this, and there is no reason any of us should come to work if we cannot.

We welcome a federal framework that sets consistent safety requirements for frontier AI. But we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence. Consistent rules to manage frontier risk so that we can maximize the benefits are a good idea (and we are excited by ideas like independent auditors).

Years ago, companies like ours developed things like Responsible Scaling Policies and Preparedness Frameworks. Those were good for that moment, and focused primarily on the deployment of completed models, not what happens during their development process.

Today's shift to focusing on safe development and evaluation will need new tools. For example, at OpenAI we now formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability, in addition to the safety work we have long done in advance of model releases.

We hope that other companies will learn from our approaches and propose their own; we think shared standards for misalignment, monitoring, and safety will lead to better outcomes. We look forward to collaborating with our colleagues across the industry to formulate the best version of these.

When we talk about “pacing”, we do not mean “stopping”. Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs.

Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.

Where we will need the help of our government is for international coordination. But first we should do what we can ourselves.

Sam Altman

There are two ways AI progress could go very badly and that we must avoid.

First, we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people. To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities.

Second, we could end up in a world with too much concentration of power. If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian.

Avoiding these two threats requires walking a narrow middle path; for example, one country could gain too much power. Another example is one lab ending up with too much power.