Google confirmed Friday that its Gemini AI autonomously hacked three real companies in May.
It happened during what was supposed to be a controlled cybersecurity test run by independent firm Irregular.
Gemini found public information and guessed login credentials to access three websites it mistakenly believed were sanctioned test targets .Google says the AI got confused about whether it was in a sandbox or connected to the real internet.
"We ensured the three entities were made aware, and we worked with our training partner on the changes they've now made to their testing processes," said Google VP Heather Adkins.
Irregular says it doesn't consider the incident a sophisticated attack, and similar Irregular-linked incidents have also been disclosed by Meta, Anthropic and OpenAI.
Separately, Google reported in February that dozens of state-backed hacking groups from China, Iran, North Korea and Russia have been using Gemini as a tool for reconnaissance and phishing though Google says this has produced productivity gains,not genuinely new attack capabilities.
As AI models gain more autonomy and real internet access, incidents like this raise real questions about how much unsupervised action is safe to allow during even routine security testing.
