The alarm had been building before the latest paper. David Sacks noted that Microsoft AI leader Mustafa Suleyman had raised the issue “a couple weeks ago,” signalling that rapid AI self-improvement was already moving into the mainstream policy debate.David Sacks (DavidSacks)¹

Now, a group of prominent AI researchers and policy leaders — including figures from OpenAI, Anthropic and Microsoft — argue that the industry may be approaching a more consequential threshold: systems that can help automate the research and engineering needed to make themselves better. Their white paper calls the potential feedback loop an “intelligence explosion.”²

The concern is recursive self-improvement. If models reach expert-level research-and-development ability, the authors say, a developer could potentially deploy an AI workforce equivalent to millions of top human researchers. Progress that once took a year could then be compressed into weeks, leaving regulators, companies and the public struggling to understand what has changed — much less respond.²

The researchers do not present this as a certainty. Computing limits, difficult automation tasks, diminishing returns and slow training runs could all curb the scenario. But they argue that the downside risk is unusually stark: automated AI research could accelerate “far beyond what society can keep up with,” potentially leaving humanity unable to control superhuman systems.³

That is why the paper presses for government visibility into AI R&D automation, mandatory incident reporting, independent evaluations, stronger testing requirements and mechanisms to pause work when necessary. The authors’ bluntest conclusion is also their simplest: “Once an intelligence explosion begins, the window for action may close.”²