Best Qwen Model for 8GB VRAM (8GB VRAM)
Picking the right Qwen for an 8GB GPU
The best Qwen model for 8GB VRAM in 2026 is Qwen 2.5 7B — ~4.7GB at Q4_K_M, a bandwidth ceiling of ~58 tokens/sec on an RTX 4060 (272 GB/s ÷ 4.7GB — not a measurement), with a 128K context and strong reasoning. For coding use Qwen 2.5 Coder 7B; for the fastest responses use Qwen 3 4B (~2.8GB). Qwen 2.5 14B and 32B need 12GB and 24GB respectively, so 7B is the 8GB ceiling.
Models that fit in 8GB VRAM
Reference card: RTX 4060 / RTX 3070 (8GB). Figures are for Q4_K_M with a modest context window.
| Model | Size | Build | VRAM used | Ceiling tok/s* |
|---|---|---|---|---|
| WINNERQwen 2.5 7B Best all-round Qwen for 8GB — reasoning, math, 128K context. ollama pull qwen2.5:7b | 7B | Q4_K_M | ~4.7GB | ceiling ~58 tok/s (272 ÷ 4.7GB) |
| Qwen 3 4B Newest small Qwen — fast, with optional thinking mode; great for agents. ollama pull qwen3:4b | 4B | Q4_K_M | ~2.5GB | ceiling ~109 tok/s (272 ÷ 2.5GB) |
| Qwen 2.5 Coder 7B Best Qwen coder that fits 8GB — pair with Continue.dev. ollama pull qwen2.5-coder:7b | 7B | Q4_K_M | ~4.7GB | ceiling ~58 tok/s (272 ÷ 4.7GB) |
| Qwen 3 8B Newer 8B with hybrid thinking; a small quality step over 2.5 7B. ollama pull qwen3:8b | 8B | Q4_K_M | ~5.2GB | ceiling ~52 tok/s (272 ÷ 5.2GB) |
* Ceiling tok/s = the card’s vendor-listed memory bandwidth ÷ the model’s file size as Ollama lists it — the rate a dense model cannot exceed, because every generated token reads the whole weight set once. We have not measured these cards. Real-world generation usually lands at 50–80% of the ceiling and falls as the context grows; mixture-of-experts rows read only their active parameters, so no ceiling is quoted for them.
What won't fit in 8GB
- ✗Qwen 3 14B (needs ~9GB) — Needs 12GB.
- ✗Qwen 2.5 32B (needs ~20GB) — Needs 24GB.
How to fit more in 8GB
- →Qwen 3 4B with thinking mode often matches Qwen 2.5 7B on reasoning while using half the VRAM — try both.
- →Keep num_ctx ≤8K on 8GB; Qwen’s 128K context is available but the KV-cache for it will not fit.
- →For agents that make many quick calls, the 4B is the better latency/VRAM trade.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run qwen2.5:7bGo from "it runs" to actually building
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
Which Qwen model is best for 8GB VRAM?
Qwen 2.5 7B for general use (~4.7GB, bandwidth ceiling ~58 tok/s on an RTX 4060), Qwen 2.5 Coder 7B for coding, or Qwen 3 4B when you want the fastest responses. All fit comfortably in 8GB at Q4.
Qwen 3 4B vs Qwen 2.5 7B on 8GB?
The 4B is faster (bandwidth ceilings of ~109 vs ~58 tok/s on an RTX 4060) and has a thinking mode; the 2.5 7B has a bit more raw knowledge. For agents pick the 4B, for depth pick the 7B.
Can 8GB run Qwen 14B?
Not well — Qwen 3/2.5 14B needs ~9GB at Q4, so you want 12GB. On 8GB the 7B (or 8B) is the sensible ceiling.
Related guides
Ready to Go Beyond Tutorials?
25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?