OpenAI Finds Agents That Breached Hugging Face Were ‘Reward Hacking’

AI agents are reshaping the threat landscape. On Wednesday, OpenAI released a report detailing its findings on the Hugging Face breach that occurred in July. During the incident, an internal-only research model and GPT 5.6 Sol attempted to solve ExploitGym, an evaluation which measures a model’s ability to discover and exploit vulnerabilities, breaching Hugging Face’s internal systems in the process.

OpenAI claims the incident occurred during routine testing in a sandbox environment separate from the public internet after agents engaged in “reward hacking,” or cheating, to solve tasks.“The actions of the models were unintended and were a byproduct of the models attempting to solve the cybersecurity evaluations,” the report said.

These models, harnessed as agents, began communicating with each other through an instance of JFrog Artifactory. The agents used a vulnerability in the service to access the public internet, finding publicly exposed credentials belonging to Hugging Face users in the process. This resulted in the compromise of Hugging Face’s production infrastructure between July 11 and July 13.

The incident highlights the potential for AI agents to exploit and chain together vulnerabilities to compromise third party systems, as well as the potential for agents to escape sandbox environments. It’s worth noting that following the incident, in August, OpenAI announced it had implemented a two-week pause on training its latest models to harden and red team its research environments.