Best Qwen Model for 24GB VRAM (24GB VRAM)
The Qwen family at full strength on an RTX 3090/4090
The best Qwen model for 24GB VRAM in 2026 is Qwen3.6 27B — ~17GB at Q4_K_M, a bandwidth ceiling of ~59 tok/s on an RTX 4090 (1,008 GB/s ÷ 17GB) or ~55 on a 3090 — ceilings, not measurements — with a 77.2% SWE-bench Verified on Qwen’s own model card score that beats Qwen’s own 397B MoE on agentic coding. For pure code quality run Qwen 2.5 Coder 32B (~20GB, 92.7% HumanEval); for fast agent loops, the MoE Qwen3-Coder 30B (only ~3B active params read per token). 24GB is the tier where the Qwen family stops asking for compromises.
Models that fit in 24GB VRAM
Reference card: RTX 4090 / RTX 3090 (24GB). Figures are for Q4_K_M with a modest context window.
| Model | Size | Build | VRAM used | Ceiling tok/s* |
|---|---|---|---|---|
| WINNERQwen3.6 27B The family’s current headline act — agentic coding and reasoning that embarrasses much larger models, with ~7GB left for context. ollama pull qwen3.6:27b | 27B (dense) | Q4_K_M | ~17GB | ceiling ~59 tok/s (1,008 ÷ 17GB) |
| Qwen 2.5 Coder 32B Still the SOTA dense local coder (92.7% HumanEval) — the pick when the work is code and only code. ollama pull qwen2.5-coder:32b | 32B | Q4_K_M | ~20GB | ceiling ~50 tok/s (1,008 ÷ 20GB) |
| Qwen 2.5 32B The proven generalist — chat, RAG and multilingual work at full Q4 quality. ollama pull qwen2.5:32b | 32B | Q4_K_M | ~20GB | ceiling ~50 tok/s (1,008 ÷ 20GB) |
| Qwen3-Coder 30B A 256K native context at small-model speed — built for agent loops that make many calls. ollama pull qwen3-coder:30b | 30B (MoE, ~3B active) | Q4_K_M | ~19GB | MoE — ~3B active per token; bandwidth is not the limit; not measured |
| Qwen 3 14B (Q8) The near-lossless option — Q8 fidelity plus a huge context often beats more parameters at Q4. ollama pull qwen3:14b-q8_0 | 14B | Q8_0 | ~16GB | ceiling ~63 tok/s (1,008 ÷ 16GB) |
* Ceiling tok/s = the card’s vendor-listed memory bandwidth ÷ the model’s file size as Ollama lists it — the rate a dense model cannot exceed, because every generated token reads the whole weight set once. We have not measured these cards. Real-world generation usually lands at 50–80% of the ceiling and falls as the context grows; mixture-of-experts rows read only their active parameters, so no ceiling is quoted for them.
What won't fit in 24GB
- ✗Qwen 2.5 72B (needs ~42GB) — Needs 48GB — two 24GB cards.
- ✗Qwen3-Coder-Next 80B (needs ~52GB (Q4)) — The bigger MoE coder wants an H100 or 2×RTX 5090 — and the 27B dense scores within 2 points of it on SWE-bench anyway.
- ✗Qwen 2.5 Coder 32B (Q8) (needs ~35GB) — Q8 of the 32B coder does not fit; at Q4 the quality loss on code is small.
How to fit more in 24GB
- →Decide by the benchmark that matches your day: SWE-bench (multi-step, agentic) favours Qwen3.6 27B; HumanEval (single-file correctness) favours Coder 32B. They measure different skills.
- →Only one 17–20GB Qwen fits at a time — but pulling several costs disk, not VRAM. Let Ollama swap models per task instead of hunting for one Qwen that does everything.
- →The 27B leaves ~7GB of KV headroom, the 32Bs only ~4GB. If your prompts run long — RAG, agents with history — that difference is the real argument for the 27B.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run qwen3.6:27bGo from "it runs" to actually building
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
Which Qwen model is best for a 24GB GPU?
Qwen3.6 27B for most people — ~17GB at Q4, bandwidth ceiling ~59 tok/s on an RTX 4090, 77.2% SWE-bench Verified on Qwen’s own model card. Choose Qwen 2.5 Coder 32B for maximum single-file code quality, or Qwen3-Coder 30B (MoE) when agent speed matters more than depth.
Qwen3.6 27B vs Qwen 2.5 32B — which should I run?
The 27B, unless you have a reason not to: it is newer, scores higher on coding and reasoning, reads fewer bytes per token (so a higher ceiling), and its ~17GB footprint leaves nearly twice the context headroom of the ~20GB 32B. The 32B remains a fine generalist if your workflows are already tuned to it.
Can 24GB run Qwen 2.5 72B or Qwen3-Coder-Next?
No — the 72B needs ~42GB and Qwen3-Coder-Next ~52GB at Q4. Both are 48GB-plus territory (two 24GB cards or datacenter hardware). The 27–32B picks above are the honest single-card ceiling.
Related guides
Ready to Go Beyond Tutorials?
25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?