Push for AI regulation mounts as talk of AI's "existential" risks go mainstream

In the past few days, I’ve heard a lot of people repeating that old saw—often wrongly attributed to Vladimir Lenin—about there being “weeks when decades happen.” It certainly seemed to be one of those weeks in AI. Concern about existential risk has been a strain of AI discourse for decades. But, despite occasionally making headlines when someone like Elon Musk, Sam Altman, or Geoffrey Hinton would express their fears about AI posing a grave risk to the species, it never really cemented itself in the general public’s consciousness in the way, say, climate change, or the risk of nuclear war, has. If politicians debated AI regulation at all, the discussions centered around data center construction and utility bills, jobs, education, mental health, algorithmic discrimination, and civil liberties, not the risk of rogue AI killing people—maybe even all the people. Until now, that is.The drumbeat of dire warnings from employees resigning from—or in some cases still working for—Anthropic, OpenAI, and Google DeepMind, all saying that the leading AI companies are developing the technology recklessly and risking human extinction, has dominated the global news cycle for an entire week (which is really saying something in this day and age.) AI company CEOs and politicians have been stirred to respond. After years in which both domestic AI regulation and efforts at some kind of international AI governance regime had mostly stalled, suddenly the air is electric with possibility.My Fortune colleague Nick Lichtenberg had a good story on why the resignation jeremiad of former Anthropic and OpenAI safety researcher Jacob Coxon had such impact when previous warnings, often from much higher-profile individuals, did not. The short answer is that coverage of the Hugging Face incident and other “rogue AI” episodes as well as people’s own experiences using AI agents seems to have opened the Overton window on discussing “loss of control” dangers. The timing, with Anthropic on the verge of an IPO and OpenAI edging closer to one too, also no doubt played a role.The question now is what happens next? Fortune editor-in-chief Alyson Shontell sat down with Altman on Friday to ask him those questions for her “Fortune 500: Titans & Disruptors of Industry” vodcast (we just call it “Titans” for short.) Altman said the company was in favor of coordinating an industry-wide slowdown in the pace of AI development with bitter rivals, including Anthropic and SpaceX, as well as Google DeepMind, Meta, and perhaps others. (Altman famously doesn’t get along with either Anthropic CEO Dario Amodei or SpaceX’s Elon Musk.) He hinted that such discussions were already underway and that a coordinated slowdown might be announced soon. He also said that, if necessary, he would have no problem telling investors that OpenAI had taken actions to prioritize safety that had cost them financially—and that OpenAI’s investors were warned of this possibility going in. He also definitively said OpenAI would not go public this year, in part due to the current concerns about the safety of the latest AI models, but also, he hinted, because OpenAI’s business isn’t yet in the right place. You can check out the full vodcast episode here. It’s well worth your time to watch.