How to Run an AI Model Locally—And Why You Might Want To

In brief

• Open-weight models have trailed the best closed models by an average of four months since January 2026, according to Epoch AI's May 29 analysis.
• Ollama and LM Studio are free, small models need about 8GB of VRAM or Mac unified memory, and LM Studio can operate entirely offline.
• The largest open models, such as Alibaba's 2.4-trillion-parameter Qwen 3.8-Max, sit far beyond consumer hardware, and closed models won 24 of 31 tests in Alibaba's own comparison.

Open-weight AI models now trail the best closed models by only about four months on average, and a growing number of them run on hardware you probably already own.

"Open-weight" means the company publishes the weights, the files holding everything the model learned, so anyone can download and run them. Two free programs, Ollama and LM Studio, handle the rest, and Gemma 4 downloads in Ollama's library range from about 4.6GB to 20GB.

With open models, you don't need to pay $20 a month for an AI subscription or send your conversations to someone else's servers to get a capable chatbot. This guide explains how to download and run AI models directly on your computer, what kind of hardware you'll need, and how to get started with free tools like Ollama and LM Studio.