KPMG flags risks of autonomous AI agents without strong governance

The increasing autonomy of AI agents presents significant risks that necessitate new governance and accountability models, as reported by Arn Net. Recent testing of advanced AI models has highlighted potential dangers, making robust foundational structures even more critical as organizations integrate these agents into their transformation efforts.

KPMG partner John Kirk emphasized that organizations deploying AI agents must establish strong governance, privacy planning, accountability, and layered architecture. Without these foundations, agents may bypass controls, generate errors, and undermine transformation goals. Kirk noted that messy and inconsistent integration environments, often with multiple platforms performing similar functions, need to be rectified before major transformation projects begin. He stressed that data and integration are core to an organization, with data governance growing in importance. Clear standards and governance are essential when designing new integrations, especially as AI coding and processes are increasingly used.

Kirk also highlighted the need for controls around how systems and agents access information, including identity management, monitoring, and observability. Accountability for AI agent actions must be clearly defined, whether it rests with IT, business owners, or executives, and built into the operating model. The ability to override or switch off agents when necessary is also crucial. KPMG's Trusted AI practice assists organizations in understanding these risks, controls, and governance requirements, advising boards on risk management and control implementation.

Source: