Why you should worry about Anthropic, OpenAI's proposed AI risk evaluators

The most important new job in artificial intelligence may come with an office badge, a company laptop, and access to some of the most closely guarded systems in technology: the large language models of AI leaders including Anthropic and OpenAI.  

What it may not come with is the power to stop them. 

Anthropic CEO Dario Amodei has proposed embedding third-party safety evaluators inside frontier AI companies on an ongoing basis as part of a plan to slow the advance of increasingly capable models, after former Anthropic researcher Jacob Coxon resigned, warning that frontier labs were racing toward systems they might not control. 

In an essay published last weekend that has led to a regulatory split between factions within the AI community and government, including President Trump, Amodei committed Anthropic to providing third-party evaluators with access comparable to internal risk teams, and the right to publish findings without the company's editorial control, subject to limited redactions. He explicitly cited embedded bank supervisors as a precedent, writing that his proposal "has precedent in the banking industry, which sometimes involves regulatory 'supervisors' embedded along with employees."