ai and ml

OpenAI pauses some training amid allegations its rogue agents behaved more badly than first thought

Amid allegations that agents may have gone off the rails thousands of times, China set up some kind of agentic incident hotline

The AI safety debate advanced at high speed over the weekend, amid new allegations that rogue agents have behaved more badly than first thought – and in greater numbers.

The fun started on Friday when OpenAI quietly disclosed it had paused training of its most advanced models.

The AI upstart buried that news in a “misalignment report” – that’s OpenAI-speak for its reports on rogue agents – titled “An agent used DNS to reach an external chatbot.”

The good news is that the agent involved in this incident never reached the open internet.

The bad news is that the agent, which was attempting to complete a search-based training task, was able to reach the chatbot due to insufficient DNS filtering in a training sandbox. Or as OpenAI put it, “a gap in our internet-access restrictions” – which was also a problem in the Hugging Face attack.

“The incident exposed a gap in our controls over network restrictions,” the report reads. “We therefore stopped the affected training run and have subsequently decided to pause all other training, evaluation, and inference with tool-use (defined broadly) for our most capable models until we have both validated that the gap is resolved and performed additional red-teaming of the system.”