Sam Altman confident AI industry can handle safety risks

"Accidents with any new technology are unavoidable," the OpenAI CEO tells a conference.

TL;DR

OpenAI CEO Sam Altman expressed confidence in the AI industry's ability to handle safety risks.
He acknowledged that accidents with new technologies are unavoidable and stressed the need for transparent reporting.
Altman suggested companies should be prepared to slow down or stop AI development if safety is compromised.
He used air travel safety as an example of how new technologies can become safer through reporting and regulation.
Social media, especially short-form video, was cited as a cautionary tale regarding the societal impact of technology.
Altman described AI as a powerful new technology, comparing it to fire, and believes the industry has only begun to explore its potential for making people more capable.

Continuar leyendo el artículo original

Elon Musk reposted

SMX 🇺🇸

Eight years ago, Elon Musk said:

“Mark my words: AI is far more dangerous than nukes.”

Elon didn’t give in to the AI doomers. In fact, his position is very different from theirs. He has warned about the risks of AI for years while still pushing for its development, with a strong focus on safety. His view is that companies should build powerful AI, but do it safely and responsibly, even if that means slowing the pace when necessary

“I think the danger of AI is much greater than the danger of nuclear warheads by a lot and nobody would suggest that we allow anyone to just build Nuclear warheads if they want that? That would be insane. And Far.”

Sam Altman

The world deserves confidence that American companies developing increasingly capable AI will act responsibly, especially as the trajectory of progress has steepened. Every frontier lab must deliver on this, and there is no reason any of us should come to work if we cannot.

We welcome a federal framework that sets consistent safety requirements for frontier AI. But we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence. Consistent rules to manage frontier risk so that we can maximize the benefits are a good idea (and we are excited by ideas like independent auditors).

Years ago, companies like ours developed things like Responsible Scaling Policies and Preparedness Frameworks. Those were good for that moment, and focused primarily on the deployment of completed models, not what happens during their development process.

Today's shift to focusing on safe development and evaluation will need new tools. For example, at OpenAI we now formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability, in addition to the safety work we have long done in advance of model releases.

We hope that other companies will learn from our approaches and propose their own; we think shared standards for misalignment, monitoring, and safety will lead to better outcomes. We look forward to collaborating with our colleagues across the industry to formulate the best version of these.

When we talk about “pacing”, we do not mean “stopping”. Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs.

Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.

Where we will need the help of our government is for international coordination. But first we should do what we can ourselves.

Sam Altman

There are two ways AI progress could go very badly and that we must avoid.

First, we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people. To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities.

Second, we could end up in a world with too much concentration of power. If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian.

Avoiding these two threats requires walking a narrow middle path; for example, one country could gain too much power. Another example is one lab ending up with too much power.

Elon Musk

People keep misunderstanding this. I CONTRIBUTED to Bostrom’s Superintelligence book and he thanks me by name in the foreword.

I was thinking about AI safety long before 2014.

David Sacks reposted

CBS News

EXCLUSIVE: Trump AI advisor David Sacks tells CBS News' @JoLingKent that tech leaders like Anthropic CEO and co-founder Dario Amodei have a responsibility to ensure their products are safe — and should "step aside" if they can't control what they're building. https://t.co/mDq88h7pCA

Elon Musk

Grok @Bot summary

Elon Musk

This is the way

Yann LeCun reposted

Sayash Kapoor

What does it mean to pace the frontier?

Over the last month, @random_walker and I have analyzed the loss-of-control incidents at AI companies to understand what technical and policy interventions can improve safety and what companies should do to pace the frontier. The result is a new 13,000 word essay — our most substantial writing on AI safety since AI as Normal Technology.

A summary of our arguments:

1) The polarization between the cybersecurity and AI safety communities is counterproductive. The safety community largely sees these incidents as a crisis for alignment, and worries that these incidents will become more damaging as agents become more capable. Cybersecurity practitioners largely see companies failing to take basic security precautions. We offer a middle ground between these communities as a way forward for improving AI safety.

2) We agree with security practitioners that OpenAI did not take adequate protections for controlling their agents. But this is not just a matter of applying 30-year-old security methods to a new domain. Security for AI agents — AI control — while important, is not a solved problem. While known control methods would have prevented the Hugging Face incident, as agent capabilities continue to advance, we will only be able to control them if we invest adequately in control interventions.

3) We also agree with security practitioners’ implicit position that these incidents are primarily a security story. In the AI safety community, rogue agents are treated as inherently catastrophic because of the assumption that there is an endless list of risks that will arise from their development. We disagree. We have long advocated that the best approach to AI safety is to identify the risks and address those specific risks. Over the last few months, it has become clear that one urgent risk is cyberoffense, because it has unique properties that allow agents to carry it out autonomously. We should similarly invest in defenses against other specific risks, such as biorisk and risks from military AI.

4) We agree with the safety community that there is an urgent need for technical and policy interventions to prevent loss-of-control incidents. But in our view, marginal investments in control are more likely to be effective compared to those in alignment. We view these incidents as illustrating the lack of emphasis on AI control within companies, despite the availability of known techniques. More broadly, there are many common-sense policy proposals that could help promote investments in AI control where we share common ground with the safety community.

5) Organization governance should be a key tool for pacing the frontier. Unfortunately, AI companies are trying to reinvent basic aspects of organizational governance as a problem to be solved by improving the technology. But even developing better control techniques will not be enough if irresponsible individuals or teams within large organizations can choose not to use them. When a single misconfigured RL environment or unmonitored evaluation can cause real-world harm, individual teams should not be able to run potentially dangerous experiments without oversight from legal, security, and other teams. AI companies need processes for reviewing experiments, assigning responsibility for monitoring them, and investigating warning signs deeply before restarting experiments. If putting these processes in place requires pausing some experiments, companies should do so.

6) How should we reason about AI's impact on cybersecurity? It's plausible that advances in agent capabilities upset the offense-defense balance for cybersecurity. We cannot yet be certain, but there is enough evidence that agent capabilities might soon make widespread cyberoffense possible that urgent action is warranted. We discuss potential interventions for tilting the offense-defense balance towards defenders.

7) How our views have evolved over the last year. We take stock of AI progress and share how we have updated our views. In the essay, we did not pay sufficient attention to safety risks that arise during development and evaluation (as opposed to the widespread deployment of models). We were too confident that companies would take basic control precautions and underplayed the importance of jaggedness, which led us to underestimate how quickly capabilities could improve in domains such as cybersecurity.

8) At the same time, many distinctive claims of AI as Normal Technology have held up. In particular, we think recent incidents support our continuity hypothesis — the behavior of "rogue" agents became apparent and widely publicized while they are still incompetent at causing serious harm or hiding their traces. The societal reaction to even the relatively small harms from these incidents has been fierce (and the safety community deserves credit for keeping up pressure on companies). Whether this translates into meaningful changes in companies’ behavior remains an open question, and a test of the usefulness of the AINT framework.

9) In short, we’ve tried to synthesize the AI safety and cybersecurity communities' views into a coherent plan of action: hold companies responsible, invest in control, and strengthen defenses against specific risks.

Yann LeCun reposted

Derya Unutmaz, MD

I completely agree with this take related to the “AI will create viruses that will kill us all” BS. But let me add my few cents, because I am really really angry!

I worked with one of the deadliest viruses in history, HIV, for two decades. I was one of the early scientists to engineer it, and engineered versions of HIV are now helping cancer patients. I understand the immune system that defends us against viruses extremely deeply. I have worked in high security biohazard labs.

I don’t have a PhD in AI, but I have been involved with AI since the early 90s and have been all in on AI for years.

People claiming that you can just make a virus in your garage, use AI to engineer it, and somehow build a virus that will kill everyone don’t know what the hell they are talking about!

Of course bioweapons are extremely dangerous. In fact, viruses and bacteria have killed more humans throughout history than almost anything else. HIV alone killed tens of millions of people, yet today, if you have access to effective medicines and take them properly, HIV is generally no longer a death sentence.

COVID-19 killed millions. Why doesn’t it kill at anything close to the same scale anymore? Not because the virus disappeared. It is still here. We developed collective defenses against it through our immune systems, prior exposure, vaccines, treatments, and better medical care.

And this is exactly the point.

The best way to fight biological threats, whether natural or synthetic, is to use AI to develop vaccines, treatments, antibodies, antivirals, and eventually engineer our immune system to create much stronger defenses against them.

By the way, it is already possible to make synthetic viruses. You don’t need some hypothetical superintelligent AI model to do that. We have had sophisticated molecular biology and genetic engineering for decades.

So how many people have actually died from synthetic viruses compared with natural viruses killing millions every year?

And if someday there really is an AI capable of making some unbelievable “supervirus,” then why the hell wouldn’t that same AI make it even easier to develop defenses against it?

Or cure diseases, for that matter?

Where is this damn superintelligence that has cured a single major disease?!

These AI companies and AI doomers keep talking like, “OMG, AI is going to become so unbelievably powerful that it could kill all of humanity!”

Then why is nobody asking the obvious question?

If your AI is already becoming this godlike, unbelievably powerful intelligence, why hasn’t it cured anything?

I mean ANYTHING.

Where is the cure for cancer?

Where is the cure for Alzheimer’s?

Where is the cure for aging?

Forget those. Where is the vaccine that prevents the common cold?

You are telling me this intelligence will soon be smart enough to engineer some magical virus capable of wiping out 8 billion people, defeating every immune system, every vaccine, every antiviral, every laboratory, every government, and the entire global biomedical community…

…but somehow it still can’t cure one disease?

I call that greatest intellectual dishonesty ever!

At least WE actually know how to cure things and save millions of lives with our supposedly stupid human intelligence compared with this future “superintelligence.”

Humans eradicated smallpox, which killed hundreds of millions of people throughout history.

We turned HIV from a almost certain death sentence into a manageable disease.

We developed vaccines in record time against COVID. If we had the same AI back then, would have saved millions more!

We engineer immune cells to kill cancer.

We developed antibiotics, antivirals, monoclonal antibodies, gene therapies, organ transplantation, and thousands of medicines.

We aren't fast enough but we did all of that with our ordinary human intelligence.

So imagine what actual superintelligence could do for medicine!

And here is the part that really makes me angry.

If you actually have, or are close to having, an AI capable of curing diseases and saving millions of lives, and you are deliberately slowing it down or refusing to make those capabilities available, then we also need to talk about the human cost of THAT decision, which you fearmongering people are going to be responsible for!

More than 150,000 people die every single day around the world.

Every. Single. Day. More than 90% from Cancer, heart disease, infections, aging. How many die because of AI? ZERO!

Talk about the people dying TODAY.

Talk about curing cancer.

Talk about stopping the next pandemic before it starts.

Talk about developing universal vaccines.

Talk about engineering our immune system so viruses become almost irrelevant.

Talk about curing genetic diseases.

Talk about reversing aging.

Talk about saving millions and eventually billions of lives.

Instead, we constantly hear this “AI WILL KILL US ALL!” crap, delivered with absolute certainty and with this smug smile on your faces in front of cameras, as though imagining a science fiction extinction scenarios.

I am actually angry about this. Each one of you causing a delay in AI advance every single day will be personally responsible for those 100 thousands deaths that could have been prevented!

Because if you truly believe intelligence is about to become this powerful, then the greatest question humanity should be asking is NOT:

“How could this intelligence hurt us?”

It should also be:

“WHY THE HELL AREN’T WE USING IT TO SAVE EVERYONE WE CAN?”

And if superintelligence really is coming, then make sure one of the first things we do with it is make humanity biologically damn near impossible to kill.

Satya Nadella

Great conversation with @chamath, @jason, @davidsacks, and @friedberg on all the work ahead when it comes to diffusing benefits of AI broadly, earning permission from communities, and ensuring AI safety and control.

Mark Zuckerberg

Last month I wrote about how we can build a positive and safe future for everyone: https://t.co/eoLGVY8yad

Every lab has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens.

The reality is:

- People won't want to use agents that are misaligned with them and that don't do what they ask, so labs have a strong natural incentive to make their models more aligned.

There is a lot of debate about slowing progress on capabilities until alignment catches up. My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models. Any lab that doesn't focus on alignment will fall behind.

- Labs face significant liability if their models cause harm, so they have a strong incentive to prevent this as well.

Meta delayed shipping Muse for several months to focus on safety and security. We didn't call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us. I'm proud of the security foundations we've built.

- Engaging independent evaluators and advisors is industry best practice. MSL already does this today in several areas because it helps produce better work. Other labs can just do this too. In general, it would be helpful for there to be a larger and more diverse ecosystem of evaluators.

- Committing the significant majority of compute towards serving people rather than racing towards recursive self-improvement is one of the best ways to ensure we develop this technology safely. Meta has made this commitment and other labs can do this as well.

I believe the key to building a positive future for everyone is maintaining the right balance of power. This is within our power to do.