Even the king of England has his hesitations about AI

King Charles hosted a private summit Thursday with some of the most prominent names in AI and the U.K. government, including Nvidia’s Jensen Huang, leaders from OpenAI and Anthropic, the U.K.’s new AI minister Kanishka Narayan, and even the head of the U.K.’s foreign intelligence service. Paolo Benanti, who advises the pope on AI, was also there.

The king’s remarks at the summit — held at Dumfries House, an 18th-century Ayrshire estate owned by King Charles’s charitable foundation — were notable because the royal family is usually careful, for constitutional reasons, about veering too deep into political issues — and AI is one of the most hot-button issues in the world right now. The king urged the leaders in the room to find a way to control AI “before it’s too late,” adding that the fast development of this technology — and its “substance” — was “both intriguing and deeply concerning.” 

“AI is already showing immense capacity to improve and save life,” he said, citing medicine and science as an example. “Yet those who have created these technologies are now increasingly warning that AI risks developing darker capacities, perhaps even to take life.” This remark came just days after an Anthropic researcher quit, writing online that AI companies were “racing to build machines that are much smarter than any human” and that humanity “may not survive this.”