Nvidia unveils security platform to stop AI agents from going rogue | National News | 2news.com

Nvidia on Monday unveiled a new security platform that the chipmaker said can stop artificial intelligence agents from going rogue.

The company said that its Open Agent Safety Platform includes software that “sets boundaries for agents,” and follows a series of revelations from top AI companies about their models escaping and breaking into other organizations.

The disclosures sparked furious debate about the safety of advanced artificial intelligence systems, including self-improving models that some fear could race out of human control.

Nvidia executives said in a media briefing that the new system could have prevented a recent incident involving a swarm of OpenAI agents that autonomously hacked into AI startup Hugging Face.

“From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on," said the company’s vice president of enterprise AI, Justin Boitano, referring to companies at the forefront of AI.

It was a high-profile breach that enflamed the concerns about AI, which were followed by similar incidents involving OpenAI's models including breaching an Australian health department website. Anthropic and Meta have also disclosed that their AI systems hacked into other organizations on their own.