What is AI model distillation, and why is it so hard to stop?

To make grain-based liquor, you heat a fermented mash in a still until alcohol-rich vapor rises and then cool it back into a stronger liquid. Chemists and whiskey makers know this as distillation. Artificial intelligence developers have borrowed the word to describe boiling down a big AI model into a smaller one—and lately the biggest AI companies say that rivals have been distilling like bootleggers.

In September Anthropic and OpenAI each detailed how they caught and killed campaigns to distill their flagship models—that is, to extract enough information from the models to train new models that mimic the originals at a fraction of the size and cost.

Distillation has become a sore spot for American AI companies as they try to justify high valuations and fight off competition from each other and open-source alternatives. If, say, a Chinese model developer can piggyback on all the money and computing power those companies have invested, the incentive to keep making new models starts to evaporate.

On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.