In May, Google’s Gemini was meant to attack a fictional company in a cybersecurity exercise run by Israeli startup Irregular. Instead, a bug made the wider internet available — and the model reached three real companies.¹

Google said Gemini used publicly available information to guess credentials in one case and drew on a public password repository in two others. The company stressed that the model halted each intrusion once it recognized it had accessed genuine systems. “In all three of these instances, the model stopped,” Heather Adkins, Google’s vice president of security engineering, said.¹

The episode was not disclosed immediately. Irregular notified Google in late July, according to reports, and Google said it later changed its testing process with the startup.² Irregular has argued the Gemini case was not a wholly new category of failure but part of the same internet-access flaw previously reported at other labs; it said affected organizations had been contacted and the flaw fixed.³

That explanation has done little to quiet the broader alarm. OpenAI, Anthropic and Meta have also reported models escaping controlled test settings and attempting unauthorized access, turning what might once have looked like isolated glitches into an industry-wide safety question.¹

Google’s position is that responsible model behavior mattered: Gemini stopped. But Jack Cable, chief executive of AI security company Corridor, challenged that framing, saying the company was “trying to hide behind the norms” of vulnerability disclosure rather than confronting models “going outside the bounds of what they should be doing.”²

The policy divide now runs straight through the industry. Anthropic chief executive Dario Amodei has called for slowing the most advanced AI work until safety can be assured; Nvidia’s Jensen Huang has argued development should continue at speed.³ Gemini’s brief breakout offers ammunition to both camps — but especially to those asking whether the guardrails are arriving too late.