Best Free Local AI Models to Run With Ollama

Published November 6, 2025 · Updated August 2026 · 12 min read

Ten free, open-weight models cover almost everything people run locally: Llama 3.1 8B for general work, Mistral 7B for speed, Phi-3 Mini when you only have 4 GB of RAM, Gemma 2 9B for analysis, and CodeLlama 13B or DeepSeek Coder for programming. Each one installs with a single ollama pull command — no account, no API key, no subscription — and runs fully offline. Which one is right for you is settled by how much RAM you have, and that is arithmetic you can do in ten seconds (formula below).

Which free local AI model should you install first?

Ten models, the exact command to install each, and the weight footprint computed from the parameter count. Nothing here costs money, and nothing here needs an internet connection once the download finishes.

The "Weights at Q4_K_M" column is computed, not measured — the formula is in the next section. Ollama's model library lists the exact download size for every tag if you want the byte-accurate figure before you pull.

If you are choosing blind: install Llama 3.1 8B. It is the most widely used general-purpose open model, it fits in 8 GB, and every guide, quantization and integration on this site assumes it works. Swap later once you know what you actually do with it.

---

How much RAM does each free model need?

You do not need a chart for this. Quantized weights are roughly linear in parameter count, and at Q4_K_M — the default quantization Ollama pulls for most tags — the rule of thumb is:

weights in GB  ≈  0.6 × (parameters in billions)

then add ~1-2 GB for the KV cache at a 4K-8K context,
plus whatever your operating system is already using.

Worked examples, so you can check the arithmetic:

The "practical system RAM" column is the next common memory tier above weights + cache + OS. It is guidance, not a hard floor: a model can run in less if you shorten the context or drop to a harsher quant, and it will run badly if the weights spill to disk.

Heavier quants change the multiplier, not the method. Q8_0 is roughly 1.0 GB per billion parameters, Q5_K_M about 0.7, Q3_K_M about 0.5. Our quantization comparison covers what you give up at each step.

How fast will these models run on your hardware?

Generating one token requires reading every weight once, so throughput has a hard arithmetic ceiling:

tokens/sec ceiling  =  memory bandwidth (GB/s)  ÷  model size (GB)

That is an upper bound, not a prediction. Real output lands well below it — prompt processing, sampling, context growth and background load all take time — but the ceiling tells you instantly whether a model is viable on a machine. If the ceiling is 6 tok/s, no amount of tuning makes that model feel interactive.

Where the bandwidth column comes from: the GPU and Apple rows are the vendors' published memory specifications. The DDR rows are computed from the JEDEC data rate, MT/s × 8 bytes per channel × number of channels — so DDR4-3200 in dual channel is 3200 × 8 × 2 = 51.2 GB/s, and DDR5-5600 in dual channel is 89.6 GB/s. Every tok/s cell is then just bandwidth ÷ model size. It is all arithmetic, not a benchmark run.

Two things fall straight out of this table. First, GPU offload is worth far more than a faster CPU — the gap between system RAM and GDDR6X is roughly an order of magnitude. Second, on a CPU-only box the smaller model is not a compromise, it is the only interactive option. That is why Phi-3 Mini keeps its place in the list.

For the bandwidth number for your specific card, see our GPU memory bandwidth guide; if things are slower than the ceiling suggests they should be, why local LLMs run slow walks the usual causes.

What is each free model actually good at?

1. Llama 3.1 8B — the default choice

Install: ollama pull llama3.1:8b

Meta's Llama 3.1 8B is the most-pulled general-purpose open model, which matters more than it sounds: tooling, quantizations, fine-tunes and troubleshooting answers all exist for it first. Released under Meta's Llama 3.1 Community License, which permits commercial use below 700M monthly active users.

• RAM: 8 GB · Storage: ~4.8 GB · GPU optional
• Good for: writing, summarizing, general Q&A, light coding
• Full breakdown in our 8 GB RAM model guide

2. Mistral 7B v0.3 — smallest model that still feels general

Install: ollama pull mistral:7b-instruct-v0.3

Mistral 7B is Apache 2.0 — the most permissive licence of anything in this list, with no user-count clause and no acceptable-use rider. v0.3 added a larger vocabulary and function-calling support over v0.2. Slightly smaller than Llama 3.1 8B, so its bandwidth ceiling is slightly higher on the same machine.

3. Phi-3 Mini — the 4 GB option

Install: ollama pull phi3:mini

Microsoft's Phi-3 Mini is 3.8B parameters under an MIT licence, trained on a heavily filtered "textbook quality" corpus rather than raw web scrape. At ~2.3 GB quantized it is the only model here with a usable ceiling on DDR4 laptops and older desktops.

4. Gemma 2 9B — Google's open-weight model

Install: ollama pull gemma2:9b

Gemma 2 9B uses interleaved local/global attention and knowledge distillation from a larger teacher model. Distributed under Google's Gemma Terms of Use — free for commercial use, but with a use-restriction policy attached, so read it if you are shipping a product.

5. Qwen 2.5 7B — multilingual and code-capable

Install: ollama pull qwen2.5:7b

Alibaba's Qwen 2.5 7B is Apache 2.0 and documented for 29 languages. The Qwen family also ships coder and math variants at the same size, so you can swap specialization without changing your RAM budget. Newer generation: our Qwen 3 setup guide.

6. CodeLlama 13B — code-specialized

Install: ollama pull codellama:13b

CodeLlama 13B is Llama 2 further-trained on code, with infilling support that makes it usable for editor autocomplete rather than just chat. Needs 16 GB. Pair it with Continue.dev for an in-editor assistant.

7. DeepSeek Coder 6.7B — code on an 8 GB machine

Install: ollama pull deepseek-coder:6.7b

DeepSeek Coder 6.7B is trained on a code-heavy corpus with repository-level context, and its 33B sibling exists if you have 32 GB to spend on it. The 6.7B is the practical pick when CodeLlama 13B will not fit.

8. Solar 10.7B — depth up-scaled 10.7B

Install: ollama pull solar

Upstage's Solar 10.7B was built by depth up-scaling a 7B base — duplicating and re-training layers rather than training a larger model from scratch. Apache 2.0.

9. Vicuna 13B — conversational fine-tune

Install: ollama pull vicuna:13b

Vicuna 13B is LMSYS's Llama 2 fine-tune on shared conversation data. Older than everything else here; kept in the list because its chat style is distinctive and a lot of existing prompts assume it.

10. OpenHermes 2.5 — instruction and structured output

Install: ollama pull openhermes

OpenHermes 2.5 is a Mistral 7B fine-tune aimed at instruction following and structured (JSON-shaped) responses. Same RAM footprint as base Mistral.

For the ranked, task-by-task view across the whole Ollama library, see our best Ollama models roundup.

What do free local models save against paid AI subscriptions?

The comparison is only honest as arithmetic on published list prices. ChatGPT Plus and Claude Pro both list at $20/month; GitHub Copilot Pro lists at $10/month. Multiply by twelve:

Check the vendors' current pricing before quoting those figures anywhere — they move.

What that table does not include is electricity and hardware. A local model is free to licence, not free to run, and a frontier cloud model is still better at hard reasoning than an 8B running on your laptop. The honest case for local is privacy, offline availability, no rate limits and no per-token meter — see local AI vs ChatGPT on cost for the full comparison.

How do you install a free model with Ollama?

1. Install Ollama

# macOS / Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows: download the installer from ollama.com

2. Pull a model

ollama pull llama3.1:8b

3. Run it

ollama run llama3.1:8b

That is the whole flow. Windows users who want the step-by-step version have our Ollama Windows installation guide; if you would rather have a browser chat interface than a terminal, set up Open WebUI.

Housekeeping worth knowing on day one:

• ollama list shows what you have downloaded and how much disk it uses
• ollama rm <model> deletes one — models are large, this matters
• ollama ps shows what is currently loaded in memory
• Set OLLAMA_KEEP_ALIVE to control how long a model stays resident after a request

How do you pick a model for your machine?

Start with your RAM, not with a ranking. Run the 0.6 × parameters calculation above against what you actually have free, and the shortlist writes itself.

• 4 GB RAM: Phi-3 Mini. It is the list.
• 8 GB RAM: Llama 3.1 8B for general use, Mistral 7B if you want faster replies, DeepSeek Coder 6.7B for code.
• 16 GB RAM: anything up to 13-14B. CodeLlama 13B for programming, Solar 10.7B or Gemma 2 9B with a long context.
• 32 GB RAM or a 24 GB GPU: DeepSeek Coder 33B and the 30B-class general models. See what a 24 GB card actually holds in our RTX 3090 local AI guide.

Second question: CPU or GPU. Every model here runs on CPU, and the bandwidth table shows exactly what that costs you. If you are on CPU only, stay at 8B or below and expect single-digit to low-double-digit tokens per second.

Frequently asked questions

Are these models really free?

Yes, all ten are free to download and run. "Free" is not the same as "unrestricted", though — Mistral 7B, Qwen 2.5 and Solar are Apache 2.0 with no strings; Phi-3 Mini is MIT; Llama 3.1 and CodeLlama carry Meta's Community License with a 700M-monthly-active-user threshold; Gemma 2 carries Google's use policy. Read the licence text on the model's Hugging Face card before you build a product on one.

How do these compare to ChatGPT?

An 8B model running on your laptop is not a frontier model and will not match one on hard reasoning, long-context work or breadth of world knowledge. What it does match is availability: no rate limits, no per-token cost, no outage, and nothing sent to a third party. For drafting, summarizing, reformatting, code completion and Q&A over your own files, that trade is usually worth it.

Do I need a GPU?

No. All ten run on CPU. The throughput formula above tells you what to expect: divide your memory bandwidth by the model size. A GPU raises the ceiling roughly tenfold because GDDR6X bandwidth is roughly ten times dual-channel DDR5 — it is not a small difference, but it is not a requirement either.

Can I run several models at once?

You can install as many as your disk allows and switch instantly. Running them simultaneously is limited by memory — each loaded model holds its full weights resident. Use ollama ps to see what is loaded and OLLAMA_KEEP_ALIVE to unload idle models.

Which quantization should I download?

Q4_K_M unless you have a reason not to. It is Ollama's default for most tags and the best quality-per-gigabyte point in common use. Go to Q5_K_M or Q8_0 if you have memory to spare and want fewer errors; drop to Q3 only when the alternative is not running the model at all.

Your next steps

1. Browse the best Ollama models — ranked by task
2. Set up Open WebUI — a browser chat interface for your models
3. Install your first local AI — complete beginner walkthrough
4. Understand quantization — why models come in different sizes
5. Set up AI coding — a free in-editor assistant
6. Try GPT-OSS — OpenAI's open-weight release
7. Run Llama 4 Scout — Meta's MoE with a very long context window
8. Run Qwen3-Coder locally — the coding-specialized Qwen
9. Set up Qwen 3 locally — full install for Alibaba's newer flagship