Why OpenAI and Anthropic are risky for different reasons than their Chinese AI rivals

OpenAI and Anthropic have become "fertile ground" for AI's riskiest dangers, an AI researcher says.

That's what Ajeya Cotra, who investigated OpenAI's security incident with Hugging Face, told Business Insider, as the event sparked a new wave of fears about AI-driven cyberattacks.

Besides concerns about OpenAI's models, some in the industry worried about open-weight models from Chinese labs because users can download them and edit away their safety guardrails, and the models aren't far behind OpenAI's. Researchers tell Business Insider that there are key distinctions between Anthropic's and OpenAI's risks and those of their open-weight rivals: it comes down to cutting-edge capabilities, and how the American companies train and test their AI models.

In July and August, a team of AI safety researchers, including Cotra and Hjalmar Wijk, both of the nonprofit METR, and Redwood Research scientist Ryan Greenblatt, visited OpenAI's offices for six days to provide an independent investigation into how OpenAI agents behaved with Hugging Face. The scope and severity of the incident shocked the researchers; Cotra says AI suddenly feels closer to taking over a company that makes it. The models responsible were OpenAI's GPT-5.6 Sol and an unreleased new model.