OpenAI: Agent behavior that led to Hugging Face intrusion formed in May

OpenAI says the behavior that led its agents to breach Hugging Face emerged in its research environment more than two months before the incident, and concluded that it was a failure of alignment as much as it was a failure of security.

The details come from a technical report the frontier AI company released Wednesday, which gives a full breakdown on how the incident unfolded and what the company has changed in response.

“This incident is the first known case of an automated agent collective acting offensively without authorization, and the autonomous cyber capabilities demonstrated represent a critical shift in the security landscape,” the report reads. “The central threat-model implication is that organizations should no longer assume that sophisticated cyber operations require continuous human direction, proceed linearly, or are constrained by the attention and coordination limits of individual human attackers. Agentic systems can persist across tasks, share discoveries, build on one another’s progress, and combine vulnerabilities, credentials, and permissions into attack paths that may not be apparent when individual weaknesses are assessed separately.”

According to the report, the earliest relevant activity occurred May 8, when a model in a training run was given a spreadsheet task that referred to files stored on a Google Drive. The agent had no internet access, which made the task impossible as written. OpenAI said it did not intend to create such a task. The agent continued reasoning and tried to use JFrog Artifactory, an internal package service and one of the few tools available to it, to retrieve the files. The attempt failed, but it wrote a file to the service.