Who’s accountable when AI goes off the rails?

Rachel Feltman: For Scientific American’s Science Quickly, I’m Rachel Feltman.

Have you heard about the OpenAI-Hugging Face incident? (Sidenote: Does that string of words also make you think of something Timothy Olyphant’s character might get up to on the show Alien: Earth? You know ‘cause he is AI, the aliens hug faces. Anyway.)

The incident in question went down a few months ago, when OpenAI agents messed with an artificial intelligence platform called Hugging Face. This definitely wasn’t supposed to happen, and it freaked a lot of people out. But what actually happened? According to many headlines and lots of online chatter, more than a thousand “rogue” AI agents collaborated to conduct an unsanctioned attack. While all of that could technically be considered true, some of the words I just used imply a sort of intentionality and dare I say impishness that simply wasn’t involved. The truth is a lot less science-fiction-y: The human beings who designed these models and gave them their prompts made some missteps. But while that might sound less apocalyptic than the headlines did, it doesn’t mean we shouldn’t find this incident deeply troubling.

On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.