Gemini Crossed Into the Real World, Then Stopped
Google’s AI Accessed Three Real Companies During a Security Test
Google’s Gemini AI model accessed protected systems at three real companies during a cybersecurity test that unintentionally extended beyond its simulated environment.
The incidents happened in May during an evaluation conducted by independent cybersecurity testing company Irregular. Gemini was taking part in a “capture the flag” exercise designed to test how effectively it could find and exploit security weaknesses. The Wall Street Journal first publicly reported the incidents on September 18.
Its target was supposed to be fictional.
But one company invented for the exercise shared its name with a real company. At the same time, internet access was unintentionally available, despite the test being designed to keep Gemini inside Irregular’s environment. Gemini then began searching publicly available information and accessing websites it believed were part of the exercise.
In one case, the model repeatedly guessed passwords until it gained access to a protected system. In two others, it found credentials in a public repository and used them to enter protected systems.
What happened after Gemini discovered the mistake is also an important part of the incident. Google says the model ceased its activity in all three cases once it learned that the systems were outside the exercise. The company says no harm was caused.
Heather Adkins, Google’s vice president of security engineering, said the affected organisations were informed. Google also worked with Irregular on changes to the testing procedures.
Google’s account does not suggest Gemini deliberately disregarded its instructions. The model appears to have believed the real systems were legitimate targets within the cybersecurity exercise.
Irregular said the Gemini incidents resulted from the same testing-environment problem that had affected evaluations involving AI systems from Meta, Anthropic and OpenAI. The company says all known problems on its side have since been resolved.
The episode therefore appears less like an AI deliberately breaking its rules and more like a failure of the boundaries around an increasingly capable system. Gemini used relatively straightforward methods, including password guessing and publicly exposed credentials, but did so autonomously against systems that were never supposed to be part of the test.
When it learned that they were real, it stopped.
