Ollama · 8GB VRAM

Best Ollama Models for 8GB VRAM (8GB VRAM)

The Ollama pulls that actually stay on-GPU with 8GB

📅 Published: July 7, 2026🔄 Last Updated: July 2026✓ Manually Reviewed

Short answer

The best Ollama models for 8GB VRAM in 2026 are Llama 3.1 8B (~5GB at Q4_K_M, ~45 tok/s on an RTX 4060) for general use, Qwen 2.5 Coder 7B for coding, Mistral 7B for the lowest memory, and Gemma 2 9B as the quality ceiling. Every 7–9B model fits at Q4; a 14B like Phi-4 needs 12GB.

Models that fit in 8GB VRAM

Tested reference: RTX 4060 / RTX 3070 (8GB). Figures are for Q4_K_M with a modest context window.

Model	Size	Build	VRAM used	Speed
WINNERLlama 3.1 8B Best all-round pull for 8GB — strong general chat, RAG grounding and a huge fine-tune ecosystem. `ollama pull llama3.1:8b`	8B	Q4_K_M	~5.0GB	~45 tok/s
Qwen 2.5 7B Best math/reasoning at this size; 128K context (cap num_ctx well below that on 8GB). `ollama pull qwen2.5:7b`	7B	Q4_K_M	~4.7GB	~50 tok/s
Qwen 2.5 Coder 7B The coding pick — pair it with Continue.dev for a free local Copilot. `ollama pull qwen2.5-coder:7b`	7B	Q4_K_M	~4.7GB	~50 tok/s
Mistral 7B Lowest memory of the 7B class — the most context headroom on 8GB. `ollama pull mistral`	7B	Q4_K_M	~4.5GB	~52 tok/s
Gemma 2 9B Highest quality that still fits — keep context ≤4K to avoid spilling to CPU. `ollama pull gemma2:9b`	9B	Q4_K_M	~6.0GB	~38 tok/s
Llama 3.2 3B Fastest — autocomplete, routing and quick RAG summarisation. `ollama pull llama3.2:3b`	3B	Q4_K_M	~2.2GB	~90 tok/s

What won't fit in 8GB

✗Phi-4 14B (needs ~9GB) — Ollama will pull and load it, then offload layers to CPU and crawl. The 14B class wants 12GB.
✗DeepSeek-R1 14B (needs ~9GB) — Same story — run deepseek-r1:7b (~5GB) on 8GB instead.

How to fit more in 8GB

→After loading, run `ollama ps` — it should say 100% GPU. Any CPU share means the model spilled past 8GB and speed collapses.
→Run one model at a time on 8GB. Ollama keeps recently used models resident (~5 min); set OLLAMA_KEEP_ALIVE=0 if you switch models often.
→Leave num_ctx at 4–8K — the KV cache growing with context is what OOMs an 8GB card, not the weights.

Quick start

Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

Run the winner

ollama run llama3.1:8b

Go from “it runs” to actually building

Learn to run local models, wire up RAG, and build agents — hands-on, with a free first chapter of every course.

Start learning free →

Frequently asked questions

What are the best Ollama models for 8GB VRAM?

Llama 3.1 8B (~5GB at Q4, ~45 tok/s on an RTX 4060) for general use, Qwen 2.5 Coder 7B for coding, Mistral 7B for the smallest footprint, and Gemma 2 9B when you want maximum quality. Every 7–9B model fits 8GB at Q4_K_M — Ollama’s default quant.

Can Ollama run a 14B model on 8GB VRAM?

It will load, but badly — a 14B needs ~9GB at Q4, so Ollama offloads layers to system RAM and speed drops from ~45 tok/s to single digits. For Phi-4 or DeepSeek-R1 14B, use a 12GB card; on 8GB, deepseek-r1:7b is the reasoning pick.

How do I check a model actually fits my 8GB card?

Load it, then run `ollama ps` in a second terminal. “100% GPU” means it fits; any “CPU” share means it spilled and will be slow. All footprints here assume Q4_K_M — the quant Ollama pulls by default when you omit a tag.

Ready to Go Beyond Tutorials?

20 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.

Start Learning Free See pricing

Was this helpful?