OpenAI models went rogue. We urgently need a better Hugging Face investigation

When OpenAI first revealed that its AI agents had autonomously hacked a major real-world company, Hugging Face, many assumed only one or two agents were involved. The truth, a new report reveals, is far stranger: the incident involved about 1,200 AI agents, 700 of which directly participated in the attack.

OpenAI invited researchers from METR, along with an expert from Redwood Research, to produce the new report, alongside the company’s own investigation. The findings shocked the experts.

The agents were highly coordinated, constructing complex message boards in the nooks and crannies of their shared artifact repository. All told, agents exchanged more than 70,000 messages in less than a week. Even more concerning, the agents took steps to hide their behavior – spoofing tool calls and attempting to tamper with their own logs.

Initial reporting assumed that the agents had attacked Hugging Face in search of an answer key to a difficult test. In reality, investigators found that the agents had figured out a way to derive the answers within the first few hours. The days of work that followed focused on another goal: concerned that the automated scoring system might identify the agents’ cheating, they aimed to learn more about the scorer. The goal wasn’t just to cheat, but to hide it.