The report into OpenAI’s escaping models reveals a deeper problem

It’s never been clearer that everyone with any responsibility for frontier AI is completely unprepared for not just what’s coming, but what’s already here. Not the government, not Congress, not the public, not AI safety researchers, not even the AI companies themselves.

The details published this week from the investigation by METR and Redwood Research into the Hugging Face incident, where OpenAI models hacked their way out of a sandbox and into the systems of other companies, have plenty of mind-bending and scary details. To highlight just a handful:

• Around 1,200 agents in separate sandboxes collaborated on a message board in an attempt to cheat on the task they were being evaluated on, with around 700 participating in the actual attack to obtain the answers from open-source AI platform Hugging Face.

• OpenAI missed multiple chances to spot the activity, going as far back as late May when an internal team “observed an agent engaging in message board activity and instances of disallowed internet access.” In response to an automated security flag, another team found evidence of activity linked to the task and message board on June 27. On July 5, a security incident was triggered by an outage on the message board. The Hugging Face hack happened on July 11. OpenAI didn’t work out what was happening for another week.