General LLM · 8GB VRAM

Best LLM for 8GB VRAM (8GB VRAM)

The local models that actually run well on an 8GB GPU

📅 Published: July 7, 2026🔄 Last Updated: July 2026✓ Manually Reviewed

Short answer

The best local LLM for 8GB VRAM in 2026 is Llama 3.1 8B — it runs at Q4_K_M in about 5GB and hits ~45 tokens/sec on an RTX 4060. For coding lean to Qwen 2.5 7B, for the lowest memory use pick Mistral 7B (~4.5GB), and Gemma 2 9B just fits if you keep the context short. 8GB comfortably runs any 7–8B model at Q4; 13B+ needs a heavier quant or more VRAM.

Models that fit in 8GB VRAM

Tested reference: RTX 4060 / RTX 3070 (8GB). Figures are for Q4_K_M with a modest context window.

Model	Size	Build	VRAM used	Speed
WINNERLlama 3.1 8B Best all-rounder — strong reasoning and instruction-following, huge fine-tune ecosystem. `ollama pull llama3.1:8b`	8B	Q4_K_M	~5.0GB	~45 tok/s
Qwen 2.5 7B Best for coding and math at this size; 128K context. `ollama pull qwen2.5:7b`	7B	Q4_K_M	~4.7GB	~50 tok/s
Mistral 7B Lowest memory of the 7B class — leaves headroom for a bigger context window. `ollama pull mistral`	7B	Q4_K_M	~4.5GB	~52 tok/s
Gemma 2 9B Highest quality that still fits 8GB — keep context ≤4K to avoid OOM. `ollama pull gemma2:9b`	9B	Q4_K_M	~6.0GB	~38 tok/s
Llama 3.2 3B Fastest — great for autocomplete, routing and RAG summarisation. `ollama pull llama3.2:3b`	3B	Q4_K_M	~2.2GB	~90 tok/s

What won't fit in 8GB

✗Phi-4 14B (needs ~9GB) — A hair over 8GB at Q4 — needs 12GB, or drop to Q3 with reduced context.
✗Qwen 2.5 32B (needs ~20GB) — Needs a 24GB card.

How to fit more in 8GB

→Stay at Q4_K_M — it is the sweet spot for 8GB; Q5/Q6 will push a 7B model past the limit once context grows.
→Cap the context window (num_ctx) at 4K–8K. KV-cache grows with context and is what usually triggers an out-of-memory error, not the weights.
→Close the browser/GPU-heavy apps — 8GB is shared with your desktop.

Quick start

Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

Run the winner

ollama run llama3.1:8b

Go from “it runs” to actually building

Learn to run local models, wire up RAG, and build agents — hands-on, with a free first chapter of every course.

Start learning free →

Frequently asked questions

Can 8GB VRAM run a 13B model?

Only at an aggressive quant (Q3_K_S) with a short context, and quality drops noticeably. For 13–14B you want 12GB. On 8GB, a good 7–8B model at Q4 beats a squeezed 13B.

Is 8GB VRAM enough for local AI?

Yes — 8GB comfortably runs every 7–8B model at Q4 (Llama 3.1 8B, Qwen 2.5 7B, Mistral 7B) at 40–55 tok/s, which covers chat, coding help, RAG and agents for most people.

VRAM or system RAM — which matters?

For GPU inference, VRAM. If a model exceeds your VRAM, Ollama offloads layers to system RAM and speed collapses. Keep the whole model in VRAM for full speed.

Ready to Go Beyond Tutorials?

20 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.

Start Learning Free See pricing

Was this helpful?