Best Coding LLM for 24GB VRAM (24GB VRAM)
The SOTA local coder finally fits — Qwen 2.5 Coder 32B
The best local coding LLM for 24GB VRAM in 2026 is Qwen 2.5 Coder 32B — ~20GB at Q4_K_M, ~27 tokens/sec on an RTX 4090, and the strongest open coding model you can run locally, rivalling frontier APIs on many tasks. Codestral 22B runs comfortably with a long context, and DeepSeek-Coder-V2 Lite is the fast MoE option. 24GB is where a truly capable local Copilot becomes practical.
Models that fit in 24GB VRAM
Tested reference: RTX 4090 / RTX 3090 (24GB). Figures are for Q4_K_M with a modest context window.
| Model | Size | Build | VRAM used | Speed |
|---|---|---|---|---|
| WINNERQwen 2.5 Coder 32B SOTA local coder — near-frontier code quality, fits one 24GB card at Q4. ollama pull qwen2.5-coder:32b | 32B | Q4_K_M | ~20GB | ~27 tok/s |
| Codestral 22B Comfortable with a full 32K context at 24GB; excellent fill-in-the-middle. ollama pull codestral | 22B | Q4_K_M | ~13GB | ~35 tok/s |
| DeepSeek-Coder-V2 Lite 16B MoE — run it at Q8 on 24GB for near-lossless quality at high speed. ollama pull deepseek-coder-v2:16b-lite-instruct-q8_0 | 16B (MoE) | Q8_0 | ~17GB | ~50 tok/s |
| Qwen 2.5 Coder 14B (Q8) 24GB lets the 14B coder run at Q8 for a fidelity bump plus a long repo context. ollama pull qwen2.5-coder:14b-instruct-q8_0 | 14B | Q8_0 | ~16GB | ~30 tok/s |
What won't fit in 24GB
- ✗DeepSeek-Coder-V2 236B (needs 130GB+) — Data-center class.
- ✗Qwen 2.5 Coder 32B (Q8) (needs ~35GB) — Q8 of the 32B needs ~35GB; use Q4 on 24GB (quality difference is small for code).
How to fit more in 24GB
- →Run Qwen 2.5 Coder 32B for chat/refactor and a 3B coder for inline autocomplete — split roles in Continue.dev.
- →At 24GB the 32B coder holds a 16K+ context — enough for real multi-file work.
- →Q4 vs Q8 makes little difference for code correctness; prefer Q4 of the bigger model over Q8 of a smaller one.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run qwen2.5-coder:32bGo from “it runs” to actually building
Learn to run local models, wire up RAG, and build agents — hands-on, with a free first chapter of every course.
Start learning free →Frequently asked questions
What is the best local coding model for 24GB VRAM?
Qwen 2.5 Coder 32B — the SOTA open coder, ~20GB at Q4, ~27 tok/s on an RTX 4090. It rivals frontier APIs on many coding tasks and is the reason 24GB is the target for a serious local Copilot.
Is a 24GB GPU worth it for local coding AI?
Yes — 24GB is the tier where the best open coder (Qwen 2.5 Coder 32B) runs at full Q4 with a usable context. Below 24GB you are capped at 14B coders; at 24GB you get near-frontier local code quality.
Qwen 2.5 Coder 32B vs Codestral 22B on 24GB?
The 32B is higher quality and the better default at 24GB; Codestral 22B is faster and holds a longer context, good for heavy fill-in-the-middle autocomplete. Many run the 32B for chat and a small model for inline completion.
Related guides
Ready to Go Beyond Tutorials?
20 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?