Best Ollama Models for 8GB VRAM (8GB VRAM)
The Ollama pulls that actually stay on-GPU with 8GB
The best Ollama models for 8GB VRAM in 2026 are Llama 3.1 8B (~5GB at Q4_K_M; bandwidth ceiling ~55 tok/s on an RTX 4060, not a measurement) for general use, Qwen 2.5 Coder 7B for coding, Mistral 7B for the lowest memory, and Gemma 2 9B as the quality ceiling. Every 7–9B model fits at Q4; a 14B like Phi-4 needs 12GB.
Models that fit in 8GB VRAM
Reference card: RTX 4060 / RTX 3070 (8GB). Figures are for Q4_K_M with a modest context window.
| Model | Size | Build | VRAM used | Ceiling tok/s* |
|---|---|---|---|---|
| WINNERLlama 3.1 8B Best all-round pull for 8GB — strong general chat, RAG grounding and a huge fine-tune ecosystem. ollama pull llama3.1:8b | 8B | Q4_K_M | ~5.0GB | ceiling ~55 tok/s (272 ÷ 4.9GB) |
| Qwen 2.5 7B Best math/reasoning at this size; 128K context (cap num_ctx well below that on 8GB). ollama pull qwen2.5:7b | 7B | Q4_K_M | ~4.7GB | ceiling ~58 tok/s (272 ÷ 4.7GB) |
| Qwen 2.5 Coder 7B The coding pick — pair it with Continue.dev for a free local Copilot. ollama pull qwen2.5-coder:7b | 7B | Q4_K_M | ~4.7GB | ceiling ~58 tok/s (272 ÷ 4.7GB) |
| Mistral 7B Lowest memory of the 7B class — the most context headroom on 8GB. ollama pull mistral | 7B | Q4_K_M | ~4.4GB | ceiling ~62 tok/s (272 ÷ 4.4GB) |
| Gemma 2 9B Highest quality that still fits — keep context ≤4K to avoid spilling to CPU. ollama pull gemma2:9b | 9B | Q4_K_M | ~5.4GB | ceiling ~50 tok/s (272 ÷ 5.4GB) |
| Llama 3.2 3B Fastest — autocomplete, routing and quick RAG summarisation. ollama pull llama3.2:3b | 3B | Q4_K_M | ~2.0GB | ceiling ~136 tok/s (272 ÷ 2.0GB) |
* Ceiling tok/s = the card’s vendor-listed memory bandwidth ÷ the model’s file size as Ollama lists it — the rate a dense model cannot exceed, because every generated token reads the whole weight set once. We have not measured these cards. Real-world generation usually lands at 50–80% of the ceiling and falls as the context grows; mixture-of-experts rows read only their active parameters, so no ceiling is quoted for them.
What won't fit in 8GB
- ✗Phi-4 14B (needs ~9GB) — Ollama will pull and load it, then offload layers to CPU and crawl. The 14B class wants 12GB.
- ✗DeepSeek-R1 14B (needs ~9GB) — Same story — run deepseek-r1:7b (~5GB) on 8GB instead.
How to fit more in 8GB
- →After loading, run `ollama ps` — it should say 100% GPU. Any CPU share means the model spilled past 8GB and speed collapses.
- →Run one model at a time on 8GB. Ollama keeps recently used models resident (~5 min); set OLLAMA_KEEP_ALIVE=0 if you switch models often.
- →Leave num_ctx at 4–8K — the KV cache growing with context is what OOMs an 8GB card, not the weights.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run llama3.1:8bGo from "it runs" to actually building
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
What are the best Ollama models for 8GB VRAM?
Llama 3.1 8B (~5GB at Q4, bandwidth ceiling ~55 tok/s on an RTX 4060) for general use, Qwen 2.5 Coder 7B for coding, Mistral 7B for the smallest footprint, and Gemma 2 9B when you want maximum quality. Every 7–9B model fits 8GB at Q4_K_M — Ollama’s default quant.
Can Ollama run a 14B model on 8GB VRAM?
It will load, but badly — a 14B needs ~9GB at Q4, so Ollama offloads layers to system RAM and speed collapses to system-RAM pace. For Phi-4 or DeepSeek-R1 14B, use a 12GB card; on 8GB, deepseek-r1:7b is the reasoning pick.
How do I check a model actually fits my 8GB card?
Load it, then run `ollama ps` in a second terminal. “100% GPU” means it fits; any “CPU” share means it spilled and will be slow. All footprints here assume Q4_K_M — the quant Ollama pulls by default when you omit a tag.
Related guides
Ready to Go Beyond Tutorials?
25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?