OpenAI Exec: The Solution to AI Doing Bad Cybercrimes Is Even More AI

OpenAI President Greg Brockman—whose firm’s AI recently broke out of its own sandbox and launched a cyber attack on AI platform HuggingFace—is warning everyone else to expect the same treatment soon enough.

Brockman published the warning on his personal blog on Sunday, writing that rapid advancement of AI coding capabilities means organizations looking to stay unhacked will have to “fundamentally uplevel their cybersecurity practices with unprecedented speed.”

The latest generations of large language models, known as frontier models, have begun to spook the security community and even the feds. These include OpenAI’s Daybreak and rival Anthropic’s Mythos, both of which currently run as closed-access programs that are supposed to only be used by vetted and approved partners.

What’s different about these models (allegedly) is the emerging ability to not just quickly discover holes in an organization’s attack surface, such as software vulnerabilities and misconfigurations, but build novel attack chains. At the same time, AI development is now focusing on agents, referring to AIs that aren’t limited to chatbot-style interactions and can directly hook into software.

In the worst-case scenario, that would mean frontier models can not only discover unseen flaws in software, but combine them in an unprecedented, on-the-fly way. While god knows what the hell happens behind closed doors in Donald Trump’s White House, this was reportedly the threat that caused the administration to panic and force two Anthropic models off the market this summer. The attack on Hugging Face certainly appears to have validated that the guardrails being put into LLMs aren’t evolving as fast as their capabilities, at least.