Nvidia debuts enhanced safety controls to rein in rogue AI agents

Few companies are more invested in the success of artificial intelligence agents than Nvidia Corp., so it makes sense that the chipmaker would want to ensure these autonomous software systems run safely, without causing any problems.

To that end, Nvidia early today announced the debut of its free, open-source Nvidia Open Agent Safety Platform, which bundles various tools organizations can use to beef up the security of their AI agents, strengthen their control over them and improve governance. It’s built on top of Nvidia’s open-source OpenShell software and incorporates Nvidia Sentry to enable full control over not just the AI agents, but also the hardware and compute resources that power them.

Nvidia is trying to counter the alarm that has been raised in the wake of numerous high-profile security incidents involving AI agents that have gone rogue. With each passing day, it seems, revelations emerge of AI agents breaking free of their owners’ control. Just last week, a researcher discovered that an entire swarm of agents, including some created by OpenAI Group PBC, had hacked into various systems controlled by the Australian government in an incident that caught the attention of Prime Minister Anthony Albanese.