Cyberattacks threaten to make open-source AI a little less open, Hugging Face exec warns

The very qualities that make open AI platforms valuable, like open doors and rich data, might also make them vulnerable to more hacks, Irene Solaiman, chief policy officer and SVP of strategy at Hugging Face said on a recent episode of the Equity podcast.

The October attack on OpenCode, which targeted users’ third-party API keys and caused financial damage, is the latest example. It follows an incident this summer in which an OpenAI agent swarm breached Hugging Face in an apparent attempt to cover its tracks after accessing answers to cybersecurity evaluations.

The risks aren’t limited to platforms that invite open community contributions. On Monday, AI infrastructure startup Fireworks, which helps developers run and train open AI models, disclosed that attackers had accessed some customers’ model weights, datasets, and billing information. The company temporarily disabled certain self-service dataset functions as it investigates. While the attack isn’t necessarily due to Fireworks’ openness, it highlights how much valuable information is concentrated within AI infrastructure providers. 

“The things I worry about making an open community targeted are also what makes it special,” Solaiman said on Equity. “We are necessarily permissive. We want the average hobbyist in their basement who doesn’t have an institutional affiliation to be able to contribute.”