Best Coding LLM for 24GB VRAM (24GB VRAM)
The 24GB coding pick is now Qwen3-Coder 30B, and the 32B dense coder still fits
The best local coding LLM for 24GB VRAM in 2026 is Qwen3-Coder 30B (`qwen3-coder:30b`, a 19GB download on Ollama) — a coding-specific mixture-of-experts model with about 3.3B parameters active per token and a 256K native context, so it generates at small-model speed; swebench.com lists it at 51.6% on SWE-bench Verified with the OpenHands agent. Qwen3.6 27B (~17.8GB) is the dense alternative: Qwen’s model card reports 77.2% on SWE-bench Verified with Qwen’s own agent, and its bandwidth ceiling on an RTX 4090 is ~57 tok/s (1,008 GB/s ÷ 17.8GB — not a measurement). Qwen 2.5 Coder 32B (~20GB) still fits and still writes good code in chat and completion, but swebench.com scores it at 9.0% as an autonomous agent, which is why it is no longer the pick. 24GB is the tier where all three fit on one card.
Models that fit in 24GB VRAM
Reference card: RTX 4090 / RTX 3090 (24GB). Figures are for Q4_K_M with a modest context window.
| Model | Size | Build | VRAM used | Ceiling tok/s* |
|---|---|---|---|---|
| WINNERQwen3-Coder 30B The coding pick for one 24GB card: built for agent loops (Cline, Aider, OpenHands), 256K native context, and swebench.com lists 51.6% SWE-bench Verified (OpenHands). At ~19GB it leaves about 5GB of a 24GB card for KV cache. ollama pull qwen3-coder:30b | 30B (MoE, ~3.3B active) | Q4_K_M | ~19GB | MoE — ~3.3B active per token; bandwidth is not the limit; not measured |
| Qwen3.6 27B A general model, not a code specialist, but Qwen’s model card reports 77.2% SWE-bench Verified with its own agent. Pick it if you want one model for code and everything else. ollama pull qwen3.6:27b | 27B (dense) | Q4_K_M | ~17.8GB | ceiling ~57 tok/s (1,008 ÷ 17.8GB) |
| Qwen 2.5 Coder 32B The strongest dense open coder by HumanEval (92.7%, Qwen2.5-Coder report) and still fine for chat and completion, but swebench.com scores it 9.0% as an autonomous agent. Fits one 24GB card at Q4. ollama pull qwen2.5-coder:32b | 32B | Q4_K_M | ~20GB | ceiling ~50 tok/s (1,008 ÷ 20GB) |
| Codestral 22B Leaves the most room for context of the larger coders on this page; strong fill-in-the-middle. ollama pull codestral | 22B | Q4_K_M | ~13GB | ceiling ~78 tok/s (1,008 ÷ 13GB) |
| DeepSeek-Coder-V2 Lite 16B MoE — 24GB is enough to run it at Q8 instead of Q4. ollama pull deepseek-coder-v2:16b-lite-instruct-q8_0 | 16B (MoE) | Q8_0 | ~17GB | MoE — ~2.4B active per token; bandwidth is not the limit; not measured |
| Qwen 2.5 Coder 14B (Q8) 24GB lets the 14B coder run at Q8 with room left for a long repo context. ollama pull qwen2.5-coder:14b-instruct-q8_0 | 14B | Q8_0 | ~16GB | ceiling ~63 tok/s (1,008 ÷ 16GB) |
* Ceiling tok/s = the card’s vendor-listed memory bandwidth ÷ the model’s file size as Ollama lists it — the rate a dense model cannot exceed, because every generated token reads the whole weight set once. We have not measured these cards. Real-world generation usually lands at 50–80% of the ceiling and falls as the context grows; mixture-of-experts rows read only their active parameters, so no ceiling is quoted for them.
What won't fit in 24GB
- ✗Qwen3-Coder-Next 80B (needs ~52GB (Q4)) — The bigger Qwen MoE coder (70.6% SWE-bench Verified in Qwen’s technical report) needs 64GB of GPU or unified memory.
- ✗DeepSeek-Coder-V2 236B (needs ~133GB (Q4)) — Data-center class.
- ✗Qwen 2.5 Coder 32B (Q8) (needs ~35GB) — Q8 of the 32B needs ~35GB; use Q4 on 24GB.
How to fit more in 24GB
- →Split the roles: Qwen3-Coder 30B for agent work, chat and refactors, and a small coder (qwen2.5-coder:1.5b or 3b) for inline autocomplete. Continue.dev and Cline both assign a model per role.
- →Set num_ctx explicitly before pointing an agent at a repo. Qwen3-Coder 30B at ~19GB leaves about 5GB for KV cache on a 24GB card; check that `ollama ps` reports 100% GPU after loading.
- →Prefer Q4 of the bigger model over Q8 of a smaller one, and do not keep two 17-20GB models loaded at once: they will not both fit.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run qwen3-coder:30bollama run qwen3.6:27bGo from "it runs" to actually building
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
What is the best local coding model for 24GB VRAM?
Qwen3-Coder 30B (qwen3-coder:30b, 19GB). It is a coding MoE with about 3.3B active parameters per token, so it generates at small-model speed, and swebench.com lists it at 51.6% SWE-bench Verified with OpenHands. If you want one model for code and general work, Qwen3.6 27B (~17.8GB) is the dense alternative, with a bandwidth ceiling of ~57 tok/s on an RTX 4090.
Is a 24GB GPU worth it for local coding AI?
Yes — 24GB is the tier where Qwen3-Coder 30B, Qwen3.6 27B and Qwen 2.5 Coder 32B all run at Q4 on one card with context to spare. Below it you are limited to Devstral Small 2 with a short context on 16GB, or the 14B coders on 12GB.
Qwen 2.5 Coder 32B vs Codestral 22B on 24GB?
Both still run, but neither is the 24GB pick any more: Qwen3-Coder 30B is. Between the two, the 32B writes better code in chat and completion (92.7% HumanEval in the Qwen2.5-Coder report); Codestral 22B at ~13GB leaves far more room for context and is good at fill-in-the-middle autocomplete.
Qwen3-Coder 30B or Qwen 2.5 Coder 32B?
Qwen3-Coder 30B for agentic work: swebench.com lists it at 51.6% SWE-bench Verified against 9.0% for the 32B, and as a MoE it reads only ~3.3B parameters per token. The 32B dense coder is still a reasonable chat and completion model if you already use it.
Related guides
Ready to Go Beyond Tutorials?
25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?