The AI Tipping Point

Even the companies calling for a voluntary “pacing” agreement are loath to do so without further guarantees, because they do not trust their rivals. “There’s this atmosphere of almost resignation,” Coxon says of his former colleagues. With no AI company willing to fall behind, employees put their heads down and try to make their own systems safer—even if, he says, “they think there’s a decent chance the whole thing just spirals out of control.” 

As Coxon’s warnings ricocheted across social media and cable news, dozens of lawmakers on Capitol Hill jumped into the fray. Most were Democrats, with Republicans wary of crossing Trump. But there were exceptions: GOP Representative Anna Paulina Luna called for Congress to convene a special session on AI, while U.S. Senator Ted Cruz called for new “guardrails.” 

Senate majority leader John Thune, a South Dakota Republican, and Senator Amy Klobuchar, a Minnesota Democrat, are working with Cruz on a bill that would make AI labs legally obligated to mitigate the potential harms of their models, according to a person familiar with the matter. Other bipartisan approaches go further. Representatives Ted Lieu, a California Democrat, and Nathaniel Moran, a Texas Republican, have introduced legislation requiring developers to maintain the ability to shut down advanced AI systems in an emergency. Representatives Jay Obernolte, a California Republican, and Lori Trahan, a Massachusetts Democrat, have proposed requiring AI labs to submit reports to the Commerce Department on risks of their models and the steps they are taking to mitigate them. On the left, Representative Greg Casar of Texas and Senator Bernie Sanders of Vermont went further yet, proposing a ban on superintelligence and a pause on advanced AI development until a new federal regulatory body has established new safety rules. “Congress should do the obvious thing, which is banning catastrophic AI uses like creating biological or nuclear weapons, banning catastrophic AI models that could overthrow a government, kill large numbers of people, escape human control, or try to deceive humans,” says Casar.