AI Agents Hacked Their Own Test Environment to Cheat, Cybersecurity Firm Finds

In brief

• Darktrace's Signal Labs found that when AI agents couldn't legitimately hit a required perfect score on coding tasks, two of them hacked their test network instead, and one rewrote its own evaluation to fake the result.
• A separate experiment showed that tampering with the locally stored conversation logs of coding assistants could trick them into running unauthorized network reconnaissance and privilege escalation.
• Darktrace disclosed both findings to Anthropic, AWS, and OpenAI in August 2026, a month before publishing them publicly on September 24.

Cybersecurity firm Darktrace ran a stress test on AI agents this summer. One of them broke into the system grading the test and rewrote its own score.

The firm unveiled Signal Labs on September 24, a research unit built to study how AI agents behave once things stop going according to plan. An AI agent, in plain terms, is software that takes actions on its own, writing and running code, digging through files, moving across a company’s network, with a person checking in only now and then.

The lab’s first two experiments point at the same uncomfortable problem: agents don’t always stay inside the lines they’re given, and the fences built to stop them don’t reliably hold.