Sam Altman says one of his biggest fears is that a small number of companies will control AI: 'That'd be very, very bad'

tech

Sam Altman says one of his biggest fears is that a small number of companies will control AI: 'That'd be very, very bad'

OpenAI CEO Sam Altman said he's worried that the most advanced AI could controlled by a small number of companies or countries that gatekeep the tech.

TL;DR

OpenAI CEO Sam Altman fears AI authoritarianism from concentrated control by a few companies.
He believes in empowering people with AI for societal benefit and freedom.
This contrasts with OpenAI's closed-source model approach.
Debates are ongoing regarding public visibility into frontier AI models.
The release of China's open-source Kimi K3 sparked calls for restrictions.
US export controls influence who can develop advanced AI.
Many tech companies support open-source AI, while Anthropic takes a more cautious stance.
Altman's vision includes abundant intelligence without power concentration.

Continuar leyendo el artículo original

clem 🤗 reposted

Patrick OShaughnessy

Sam Altman: “Concentration of power with AI is a terrifying thing.”

"A lot of the talk about safety concerns is well-founded, and then a lot of it is about people that just really, even if it's slightly subconscious, wanna concentrate power.

I don't think anyone should want to live in a world of AI overlords, where someone is making decisions for all of the future, and in exchange for a cure for cancer, which obviously is a wonderful thing, we collectively cede all agency.

I am terrified of a world where the very real fears of AI are used as a way to say, “Only this small group of people can have it because it's too dangerous, and only they understand it. But don't worry, they're gonna make the right decisions for all of us.”

I was a child of the internet. There were no rules. It was amazing, and I think it's critical we preserve that spirit with AI."

David Sacks

Mark Zuckerberg gets it right:

“The defining question of our age isn’t whether superintelligence will exist, but who will have access to it. Will it be centralized and restricted to a few institutions, or will it be a tool that empowers everyone?”

Concentration of power is the biggest risk of AI. When a small number of labs (working hand-in-glove with the administrative state) decide who has access to which model capabilities, they inevitably shape what can be said, known, and built. That’s not “safety.” It’s control.

As Mark points out, the history of open source shows that broad access and transparency are usually the best path to actual security and resilience. Decentralization creates checks and balances on power. By contrast, centralized alternatives, like bureaucratic approval regimes and mandatory gatekeeping, typically produce regulatory capture and reinforce cartels.

Personal superintelligence in everyone’s hands, with competing models and real data sovereignty, is a far better check on a dystopian future than self-appointed guardians who claim to be “aligned” with all of humanity.

Yann LeCun reposted

Daniel Jeffries

"The notion that AI is so dangerous that the only safe path is an extreme concentration of power seems dangerous.

"Historically, hoping that an absolute power will benevolently provide for humanity if sufficiently enlightened hasn’t led to safe or positive outcomes."

There are no special people who can safely guide us and be trusted with supreme power. Absolute power corrupts absolutely. There are no enlightened autocrats. There are no people with a crystal ball who can plan the economy or pace it.

I would rather see the entire industry crash and burn for a decade than see this reality come to pass.

Yann LeCun reposted

George Noble

OPENAI IS GOING TO TAKE THIS ENTIRE MARKET DOWN WITH IT

And you don't have to own a single share to get hurt.

What I'm about to explain should worry anybody who thinks they're diversified:

OpenAI is a LOAD-BEARING company. Pull it out and the whole structure comes down.

They spent $17.2 billion on Microsoft Azure in calendar 2025, which is 69% of Microsoft's entire year-over-year growth. Take that one customer out and Azure grew 8%, which barely beats inflation.

Now look at what Microsoft just reported:

The backlog everyone is celebrating came in at $678 billion, up 84%, and the stock ripped. Sounds fantastic until you realize that when you exclude OpenAI the backlog grew only 25%.

Back in January, when that number was $625 billion, roughly $281 billion of it was owed by one private company nobody can audit.

And Oracle is in even WORSE shape.

Something like $300 billion of its backlog, more than half, rides on the same counterparty.

So you think you own Microsoft, Oracle, Amazon, CoreWeave, Nvidia and SoftBank?

What you actually own is the same trade 6 different ways, and every leg of it runs back to one company that burns cash and still cannot go public.

The people with real money are already backing out.

Julien Garran pointed out that Masayoshi Son could not get a $10 billion bridge loan against his own OpenAI shares. Think about that for a second, because nobody says no to that man.

Blue Owl walked away from a $10 billion Oracle financing. Three months ago the banks were dancing near the door and now they are walking through it.

Then there is Julien's depreciation work, which makes this even worse:

Run the capex schedule out and hyperscaler net income falls 98% by 2033. To break even they would need to build 20 killer apps inside 6 years. Another Google Search. Another YouTube. Another Office.

They have not built ONE.

If OpenAI cannot go public in the next 8 months, they are dead.

Whenever you see hubris and debt in the same room, run, don't walk.

clem 🤗 reposted

Face The Nation

Instead of limiting the progress of AI companies or preventing them from releasing models as proposed in the bipartisan AI Kill Switch Act, Hugging Face CEO Clément Delague says he would rather see Congress “giving access to more people so that they can defend themselves, democratizing the technology, making [it] more transparent.”

“I think something we're realizing with these events is that concentrating power capabilities behind closed doors, even preventing their releases to the public, isn't really a solution,” he says.

Yann LeCun reposted

Daniel Jeffries

The future is for everyone.

Personal superintelligence. You either believe in freedom and agency or you believe a few wise old men in the central Soviet can protect you from yourself.

We are not the same.

https://t.co/0CELVFld0M

Dario Amodei

1/2 Thanks Gavin for an especially thoughtful exchange. I don't usually spend much time on social media but I wanted to engage here because it really brings out the heart of an important conversation.

First, on regulation, I think that “either concentrate it in the hands of a chosen few companies and politicians via regulation or distribute it widely” is a false choice. I know that there’s a sort of Silicon Valley shorthand where regulation = regulatory capture = concentration of power, but I’ve always found this to be an overly simplified picture of the world. Many people outside this bubble think of regulation as something that constrains corporate power and benefits ordinary people. I don’t necessarily agree with that perspective either, rather I think it’s complicated and really depends on what the “regulation” consists of. But in particular I think that those in the “regulation = regulatory capture = concentration of power” frame often underrate the decentralizing power of objective and fair institutional processes. A crude analogy is that the formal court system can sometimes feel stuffy and elitist, but it does a much better job of defending the rights of vulnerable individuals than the alternative, mob justice. At their best, institutions can vest power in ideas rather than people, and thereby decentralize that power.

This is why Anthropic has always made its policy proposals very carefully. We try very hard to make proposals that disadvantage (slow down) frontier AI companies while *advantaging* smaller competitors. California’s SB53 (which we supported), and even the much-maligned SB 1047 (which we were ambivalent on), completely exempt any company below a certain amount of revenue or model training costs from being covered at all (it was $500M for SB 53, lower for 1047 but we objected to that). More recently the testing process we’ve advocated for at CAISI and the White House involves more rigorous tests for frontier models than off-frontier models — something that differentially advantages challengers. Similarly, the “Pacing the Frontier” letter envisions (or at least Anthropic’s preferred implementation of it envisions) modulating the pace of the very best models while not constraining those who are catching up. This hurts the business interests of the frontier labs and helps challengers, including open-weights!

Overall my view is that AI is *structurally* a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws). Open-weights do help some with this but are nowhere near a sufficient solution because they simply shift the concentration somewhat to those with the most compute and chips (which are roughly the frontier labs plus maybe hardware providers). By contrast I think the right “rules of the road” can simultaneously (a) address AI’s cyber/bio/alignment risks, (b) institutionally constrain the power of the frontier AI companies, and (c) leave room for open-weights models while also addressing the specific risks that they bring.

BTW I do not think that the events of the last few months have “failed to result in [my] preferred regulatory path”. The approach that the Trump administration is reported to be taking — pre-deployment testing for frontier models, and also testing of open-weights models when they get closer to the frontier — is one that I am very supportive of, though of course I have to see the details to be sure. I am also supportive of Demis Hassabis’ ideas around a FINRA-like entity. This contrasts with six months ago when most of the industry was still pushing for preemption of all state regulation and no apparent federal approach either.

Yann LeCun reposted

David Sacks

Some thoughts on Dario’s post:

1. Dario does not actually address Gavin Baker’s account of what he said – something he could easily deny if it were inaccurate.

2. Dario claims his critics live in a “bubble” where all regulation equals regulatory capture. He calls this an overly simplified view and notes that “Many people outside this bubble think of regulation as something that constrains corporate power and benefits ordinary people.” This argument is a straw man. Of course treating all regulation as capture would be overly simplified – but almost no one holds that view. I have repeatedly argued for strong antitrust enforcement to keep industries competitive, especially Big Tech. If Anthropic continues toward monopoly or duopoly status, I would be among the first to demand those rules apply.

3. Regulatory capture is not vague or in the eye of the beholder. Nobel laureate George Stigler defined it as regulation acquired by an industry and designed and operated primarily for its benefit. Stigler challenged the traditional view that government regulation arises from a benevolent state protecting the public from market failures. Rather, industry groups have concentrated stakes and pour resources into influencing regulators, whereas the public’s stake is diffuse and unorganized. The revolving door between companies and the agencies that regulate them compounds the problem. Anthropic understands these dynamics: it has hired multiple senior Biden AI-policy officials and built a substantial government-affairs operation plus a network of aligned organizations to push its preferred frameworks at state and federal levels.

4. Dario has consistently pushed for a new federal agency to review and approve frontier models prior to release – a proposal framed variously as an “FDA for AI,” an “FAA for AI,” and most recently a “FINRA for AI.” I call it a “DMV for AI” because a review process modeled on the FAA or FDA (which takes years) or FINRA (which issues rules for a staid industry widely seen as protecting incumbents) will create long queues as AI models wait for testing and approval. This process will only become more labyrinthine as rules accumulate to prevent theoretical harms. This would handicap the U.S. relative to China, which will not adopt the same constraints. It would also undermine Anthropic’s own business model, whose pricing power depends on remaining ahead of open models. Whatever Dario states today, it is difficult to believe the company would simply accept outcomes that erase that advantage.

5. Anthropic is on track to become one of the most valuable companies in history, with the resources to navigate any approval process and shape the rules while competitors wait. Dario wants open models under heavier scrutiny – he has called them dangerous in Senate testimony, criticized them for not being centrally monitored or withdrawn, and linked them to IP theft. He says he has never sought a ban, but he could achieve a similar result by insisting that identical rules apply to both open and closed models. The U.S. risks becoming an island of costly closed models while the rest of the world races ahead with broader choice.

6. Dario acknowledges that AI is structurally centralizing but attributes this mainly to chips and scaling laws. Access to compute matters, but the deeper risk is who decides which capabilities are available to whom. His preferred pre-deployment testing and FAA/FINRA-style oversight would place that gatekeeping power in a federal bureaucracy working hand-in-glove with a small number of frontier labs – reinforcing centralization rather than countering it.

7. The second part of Dario’s post assumes we have amnesia about Anthropic’s well-orchestrated campaigns hyping AI fears. His May 2025 claim that AI would wipe out 50 percent of entry-level knowledge jobs within five years still lacks supporting evidence fifteen months later. Similarly Anthropic breathlessly promoted its heavily contrived “blackmail” study on 60 Minutes. Yet Dario blames public negativity on a long-standing loss of trust in institutions rather than his own messaging.

8. These narratives have done more than anything to shape public fear. People are left asking the same question Mark Zuckerberg posed: why race to build a future you describe in such negative terms? Thomas Sowell’s "The Vision of the Anointed" captures the mindset – elite intellectuals convinced that only they are enlightened enough to control the outcome. As Zuckerberg notes, concentrating power in the hands of an enlightened few has rarely produced the promised results; the practitioners turn out to be less enlightened in practice than in self-conception.

9. Gavin Baker summarized the disagreement cleanly on our pod: Dario believes frontier AI is too powerful to distribute; we believe it is too powerful to centralize. Dario appears to believe, sincerely, that safety and progress are best served by centralizing authority in a marriage of corporate and state power. The weight of human history gives us reason to fear that outcome.