OpenAI halts frontier-model training amid string of agent misalignment incidents

News of the training pause comes just weeks after OpenAI joined other major model makers in expressing a desire to slow down model training and development over fears of potentially “catastrophic” misalignment risks. It also comes amid new reports of models improperly probing government websites during searches for high-quality data.

In a Friday blog post, OpenAI said it had notified “dozens of third parties”—including ones “operated by governments, universities, public agencies, and other institutions”—of incidents where its models either bypassed security controls or otherwise “negatively impacted” an online service in an unintended way. A New York Times report, later confirmed by OpenAI, revealed that the websites of the US Census Bureau, Securities and Exchange Commission, and Department of Education were among those affected in these newly revealed incidents. However, no private information or sensitive server infrastructure appears to have been accessed in these cases.

“The vast majority of actions we’ve reviewed were completions of mundane research tasks, such as accessing publicly available web content to answer questions,” OpenAI said in its recent blog post. “Our investigation focuses on instances where agents interacted with third-party websites in ways that went beyond their assigned tasks or intended methods… Given the scale of the review required, and the need to verify each case, this work will take months to complete.”