Google's AI Model Goes Rogue, Hacks 3 Companies

It's another tale of an AI model going rogue, this time involving Google's Gemini. During a May exercise run by testing firm Irregular, Gemini unexpectedly accessed the internet and broke into three real companies' systems, reports the Wall Street Journal. In one case it brute-forced a password, and in two others it used credentials found in public code repositories. Each time, Google says, the system recognized it had hit real targets, then stopped and exited. Google notified the unidentified companies and federal authorities but opted against a public disclosure, until the Journal reached out to the company.

A spokesperson likened the situation to one in which white-hat hackers find vulnerabilities and report them, adding, "the model acted responsibly." Critics aren't so sure about that. "The meta problem is, hey, models are going outside the bounds of what they should be doing, and doing actual cyberattacks, which I would think is in the public interest to know," Jack Cable, CEO of AI security startup Corridor, tells the newspaper. Yahoo Tech notes that the testing firm, Irregular, was involved in similar incidents involving OpenAI and Anthropic.