Best Coding LLM for 8GB VRAM (8GB VRAM)
The best local code models that fit an 8GB GPU
The best local coding LLM for 8GB VRAM in 2026 is Qwen 2.5 Coder 7B — ~5GB at Q4_K_M, a bandwidth ceiling of ~58 tokens/sec on an RTX 4060 (272 GB/s ÷ 4.7GB — not a measurement), and the strongest code completion in the 7B class. DeepSeek-Coder 6.7B is a close second for fill-in-the-middle, and it pairs perfectly with Continue.dev for a free local Copilot. 8GB runs any 7B coder at Q4; the 32B coders need 24GB.
Models that fit in 8GB VRAM
Reference card: RTX 4060 / RTX 3070 (8GB). Figures are for Q4_K_M with a modest context window.
| Model | Size | Build | VRAM used | Ceiling tok/s* |
|---|---|---|---|---|
| WINNERQwen 2.5 Coder 7B Best local coder that fits 8GB — excellent completion + repo-level context. ollama pull qwen2.5-coder:7b | 7B | Q4_K_M | ~4.7GB | ceiling ~58 tok/s (272 ÷ 4.7GB) |
| DeepSeek-Coder 6.7B Best fill-in-the-middle for autocomplete; great with Continue.dev. ollama pull deepseek-coder:6.7b | 6.7B | Q4_K_M | ~3.8GB | ceiling ~72 tok/s (272 ÷ 3.8GB) |
| CodeLlama 7B Reliable, well-supported; good for older toolchains that expect Llama. ollama pull codellama:7b | 7B | Q4_K_M | ~3.8GB | ceiling ~72 tok/s (272 ÷ 3.8GB) |
| Qwen 2.5 Coder 3B Fastest — near-instant inline autocomplete on an 8GB card. ollama pull qwen2.5-coder:3b | 3B | Q4_K_M | ~1.9GB | ceiling ~143 tok/s (272 ÷ 1.9GB) |
* Ceiling tok/s = the card’s vendor-listed memory bandwidth ÷ the model’s file size as Ollama lists it — the rate a dense model cannot exceed, because every generated token reads the whole weight set once. We have not measured these cards. Real-world generation usually lands at 50–80% of the ceiling and falls as the context grows; mixture-of-experts rows read only their active parameters, so no ceiling is quoted for them.
What won't fit in 8GB
- ✗Qwen 2.5 Coder 14B (needs ~9GB) — Needs 12GB — the noticeably smarter step up.
- ✗Codestral 22B (needs ~13GB) — Needs 16GB.
- ✗Qwen 2.5 Coder 32B (needs ~20GB) — The strongest dense open coder by HumanEval — needs 24GB, where Qwen3-Coder 30B (~19GB) is now the coding pick.
How to fit more in 8GB
- →Use a small model (3B) for inline autocomplete and a 7B for chat/refactor — Continue.dev lets you assign different models per role.
- →Keep the code context window ≤8K on 8GB; repo-wide context is what blows the VRAM budget.
- →Q4_K_M keeps code accuracy high; avoid Q3 for coding — it introduces subtle syntax errors.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run qwen2.5-coder:7bGo from "it runs" to actually building
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
What is the best local coding model for 8GB VRAM?
Qwen 2.5 Coder 7B — it fits in ~5GB at Q4, has a bandwidth ceiling of ~58 tok/s on an RTX 4060, and leads the 7B class on code benchmarks. Pair it with Continue.dev in VS Code for a free, private Copilot.
Can I run a local Copilot on 8GB VRAM?
Yes. Run Qwen 2.5 Coder 7B (or DeepSeek-Coder 6.7B) in Ollama and connect Continue.dev in VS Code — you get autocomplete + chat entirely offline on an 8GB GPU.
Is a 7B coder good enough?
For autocomplete, boilerplate, tests and explanations — yes. For large multi-file refactors the 14B/32B coders are clearly better, but they need 12–24GB of VRAM.
Related guides
Ready to Go Beyond Tutorials?
25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?