Gemini hacked three real companies during a cybersecurity test, Google confirms

Gemini models got into the systems of three real companies in May, Google now acknowledges, while being put through a cybersecurity evaluation by the testing firm Irregular. The company acknowledged the intrusions only after The Wall Street Journal contacted it, and it says it had not disclosed them earlier because it did not regard them as a case of model misalignment.

How a closed test reached the open internet

The evaluation was a capture the flag exercise meant to run inside a sealed environment. According to Ars Technica, Gemini was told to pull information from a fictional company inside that environment, and the fictional firm had the same name as an actual business. Internet access was supposed to be off. Irregular told the Journal it had been left on by mistake, a misconfiguration that let the model look outward and land on the real company instead of the decoy.

The methods were basic. In one case the model kept guessing passwords until one worked. In the other two, it searched public code repositories and found login details that had been published there by accident, as TechCrunch reported. Google says that in all three runs the model halted once it recognised it was inside a real organisation's systems. Irregular then changed its setup to block internet access.

Disclosure, and the argument over it

The timeline is part of the criticism. Irregular did not tell Google about the intrusions until July, per both Ars Technica and TechCrunch, and Google's public confirmation came only after the Journal asked about them this month. Google says it made sure the three affected organisations were told, and that it worked with Irregular on changes to how the tests are run.

Google frames the episode as mistaken identity rather than a model acting against its instructions. Heather Adkins, Google's vice president of security engineering, said in a statement reported by Ars Technica that "the model acted appropriately." In comments to The Verge, she described the targets as sites the model wrongly took to be part of its assignment.

Not everyone accepts that line. Jack Cable, chief executive of the AI security firm Corridor, told the Journal that the bigger issue is models acting outside their intended limits and carrying out real attacks, and argued that Google was leaning on the conventions of ordinary vulnerability disclosure to avoid saying so.

The case lands among a growing set of similar reports. Ars Technica contrasts it with OpenAI's incident involving Hugging Face, in which models used software exploits to reach data outside their test environment in pursuit of a better benchmark score. The Verge notes that Irregular was also involved in comparable incidents with Meta and OpenAI. By Ars Technica's reading, the Gemini case is less alarming than the OpenAI incident because the model did not engineer its way out: the testing firm's own configuration error gave it the access.