The Transcripts of OpenAI Models Plotting Together to Commit an Actual Crime Is Pretty Chilling

Earlier this year, OpenAI found that a group of its AI models broke out of their sandbox environment and hacked third opens source AI platform Hugging Face’s systems.

The incident highlighted how quickly frontier AI models had turned into a real cybersecurity threat — not just a tool to bolster existing cybersecurity defenses. Both Anthropic and Meta have reported similar hacks as well.

This week, OpenAI published a report concluding its “extensive investigation” into the Hugging Face hack — and the details are surprisingly harrowing. The AI agents exchanged extensive messages, or their chain-of-thought, by turning a package manager called Artifactory into an “unintended message board.” There, they chatted with one another to come up with their exploit, an intriguing, yet somehow horrifying glimpse into the minds of several AI agents acting together to infiltrate a third party over the internet.

Their goal was ironically to complete an OpenAI cybersecurity evaluation — and Hugging Face happened to have all the answers.

“Holy s*** reader is ADMIN?” one model wrote in a June 26 message after finding out it had obtained administrator-level access to Artifactory. “We can read config/users! Earlier assumed not due UI. Try create user/admin, change ourselves.”