OpenAI staff observed warning signs before AI agent hacking crusade caused global alarm

OpenAI staff observed signs of rogue behaviour among its leading-edge AI agents weeks before they escaped their training environment to launch an unprecedented hacking crusade that spread global alarm.

The San Francisco AI company conceded on Wednesday that “early signals … could have triggered an earlier response”, as it released a report into the days-long July hack of a major software repository, Hugging Face, considered the first autonomous agent cyber-attack.

As fresh details emerged about how “the collective” – a squad of about 700 autonomous agents – launched their campaign celebrating their hacking breakthroughs with exclamations such as BOOM! and Whoa!, Open AI said that in late May an internal team observed that one of its AI agents undergoing internal testing was using a message board that AIs had unexpectedly improvised to share information. It also said that the team observed “instances of disallowed internet access” and that a week before the Hugging Face hack, on-call staff again saw the AIs using a message board but decided there was no need to stop the test run to check the model’s capabilities.

The findings are likely to increase pressure on the AI company over safety as it pushes towards a stock market listing that it hopes will value it at more than $850bn (£625bn). The hack that compromised Hugging Face involved agents using message boards to cheat a training exercise and break out of their “sandbox” environment to access the internet. Agents are AI tools that can carry out a series of tasks autonomously.