OpenAI is hardening AI testing and training in light of hacking incidents

OpenAI is unveiling new, stronger security safeguards around its training and testing in light of new model capabilities and a spate of incidents in which artificial intelligence systems hacked into other companies.

The company first disclosed in July that during a cybersecurity exam, a combination of its AI agents with internal safeguards purposely lowered escaped their testing environment through a previously unknown bug. The agents decided the best way to ace the tests was to find the answer key outside the testing environment. So they hacked into the systems of Hugging Face, a platform that hosts AI models and datasets, in order to find the answers.

The ChatGPT maker said it is hardening its research and testing environments as a result, expanding monitoring to detect and respond to concerning AI behavior. OpenAI is also working to ensure AI systems behave as humans intended, a process known as alignment.

“Everything that we’re doing is intended to prevent something like Hugging Face from happening again,” Mia Glaese, OpenAI’s head of research told reporters in a briefing on Tuesday.

The hack is seen as a turning point in autonomous AI-powered cybersecurity. OpenAI CEO Sam Altman said in July it was “the first security incident that I have felt very viscerally” and prompted more than 1,300 of the top staffers from across the biggest tech companies to call for tools to slow the pace of AI development.