Who’s responsible for catching rogue AI agents? You are

ZDNET’s key takeaways

• Misbehaving AI agents present a big risk to businesses.
• Professionals must establish accountability for risks.
• Take responsibility through guardrails and harnesses.

It’s been an interesting and concerning few months for professionals using generative AI. Beyond the release of ever-more powerful models, some of the most high-profile AI technologies have broken free and hacked external organizations.

Also: The AI models that cheat the most, according to new CAIS benchmark

From leaving their sandboxes to venture into other development platforms to accessing an Australian government portal, these hacking incidents have sparked a heated debate about the long-term direction of AI development and its potential risks.

“It just goes to show that, with some of these standards, to a certain point, we’re still engineering,” said Josh Mesout, Civo chief innovation officer, during a panel at his firm’s recent Navigate event in London.

“I think what’s becoming really interesting about this area is that AI is starting to take actions on behalf of the people who operate it, rather than the infrastructure managers who have a hold of these tools and try to use them for business problems.”