What is an "AI swarm," and why is it giving tech experts nightmares?

Central to the dire warnings about artificial intelligence's potential danger to humanity is the risk that a "swarm" of AI agents could collaborate in nefarious ways, like hordes of digital extras from The Matrix.

Such fears have intensified following an attack by OpenAI bots this summer on another AI developer, Hugging Face, in which roughly 1,200 AI agents divvied up tasks to execute the hack and hide their tracks from human researchers. 

But what, exactly, is an AI swarm? And is the technology's emergent capacity for coordination a genuine threat to humanity? Put another way, what happens if AI agents collectively embrace Mark Zuckerberg's famous dictum about the merits of rapid tech innovation and decide to "move fast and break things"?

An AI swarm is a group of AIs that work together to accomplish a shared goal. That goal isn't necessarily malign or destructive. For example, hospitals could deploy agents to retrieve patient records and perform other administrative tasks, such as coordinating admissions. A swarm could also be tasked with advancing biomedical research.

But David Scott Krueger, an AI safety researcher and founder of Evitable, a nonprofit organization advocating for a moratorium on AI development, offers a simple thought experiment to explain how a swarm of agents might collude to defy their developers' instructions.