OpenAI's chief scientist says AI labs may need to slow down: 'No one is prepared for the consequences'

Days after releasing a new, highly capable model, OpenAI's chief scientist is calling for a slowdown.

In a lengthy blog post on Sunday, Jakub Pachocki said he was concerned that "no one is prepared for the consequences of a continued rapid rise in machine intelligence."

He said that although OpenAI is pursuing internal technical solutions to better control powerful AI agents, "broader interventions are required." He specifically cited concerns that increasingly autonomous agents could learn to evade human oversight, break into computer systems, and trick people to accomplish their objectives.

He called for "mandated safety bars" that he said could be enforced by "a network of third-party auditors, by government agencies or by international bodies."

Sam Altman, the CEO of OpenAI, reposted Pachocki's essay on X, calling it "an important post."

OpenAI on Thursday unveiled its newest model, Astra. The ChatGPT maker said that despite Astra's unparalleled capabilities in mathematics and computer use, the model is its most aligned, meaning it has less proclivity to go rogue.

Anthropic, OpenAI's chief competitor in the field of highly advanced AI systems, has long called for more standardized government regulation. Recently, Pachocki joined those calls, signing an open letter in July asking the federal government to pace AI development.