Best Ollama Models for 24GB VRAM (24GB VRAM)
What an RTX 3090/4090 really runs in Ollama — 27B dense to full 32B
The best Ollama model for 24GB VRAM in 2026 is Qwen3.6 27B — a dense 27B at ~17GB Q4_K_M and ~30 tok/s on an RTX 4090, which beats far larger models on agentic coding (68.9% SWE-bench Verified). 24GB also runs the full 32B class: Qwen 2.5 Coder 32B (~20GB) for code, DeepSeek-R1 32B for reasoning. A 70B still needs ~40GB.
Models that fit in 24GB VRAM
Tested reference: RTX 4090 / RTX 3090 (24GB). Figures are for Q4_K_M with a modest context window.
| Model | Size | Build | VRAM used | Speed |
|---|---|---|---|---|
| WINNERQwen3.6 27B The current headline pick — beats Qwen’s own 397B MoE on agentic coding (68.9% SWE-bench Verified) with room left for context. ollama pull qwen3.6:27b | 27B (dense) | Q4_K_M | ~17GB | ~30 tok/s |
| Qwen 2.5 Coder 32B The SOTA dense local coder (92.7% HumanEval) — the reason serious builds target 24GB. ollama pull qwen2.5-coder:32b | 32B | Q4_K_M | ~20GB | ~27 tok/s |
| Qwen 2.5 32B The long-standing 24GB default for general chat, RAG and multilingual work. ollama pull qwen2.5:32b | 32B | Q4_K_M | ~20GB | ~28 tok/s |
| Qwen3-Coder 30B MoE agentic coder with a 256K native context — 30B-class reasoning at small-model speed. ollama pull qwen3-coder:30b | 30B (MoE, ~3B active) | Q4_K_M | ~18GB | ~50 tok/s |
| DeepSeek-R1 32B Best local reasoning model — visible chain-of-thought at full 32B quality. ollama pull deepseek-r1:32b | 32B | Q4_K_M | ~20GB | ~30 tok/s |
What won't fit in 24GB
- ✗Llama 3.3 70B (needs ~40GB) — Needs 48GB (2×24GB). On one card it only runs at Q2 with heavy offload — a 32B at Q4 beats it in practice.
- ✗gpt-oss:120b (needs ~65GB+) — Workstation class — a single 80GB GPU or a 64GB+ Mac with offload.
How to fit more in 24GB
- →Qwen3.6 27B at ~17GB leaves ~7GB spare — spend it on a 32K context, or step up to a Q5/Q6 pull when you want maximum fidelity from one model.
- →The 32B models (~20GB) leave only ~4GB for KV cache — set OLLAMA_FLASH_ATTENTION=1 to hold a 16–32K context without spilling to CPU.
- →Pair one dense 27–32B with qwen3-coder:30b — the MoE covers fast agentic loops at ~50 tok/s while the dense model handles deep work (mind OLLAMA_KEEP_ALIVE: both loaded at once will not fit).
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run qwen3.6:27bGo from “it runs” to actually building
Learn to run local models, wire up RAG, and build agents — hands-on, with a free first chapter of every course.
Start learning free →Frequently asked questions
What can an RTX 3090 run with Ollama?
Everything up to a dense 32B at Q4: Qwen3.6 27B (~17GB), Qwen 2.5 Coder 32B and DeepSeek-R1 32B (~20GB each) all fit its 24GB. The model list is identical to an RTX 4090 — the 4090 is just faster — which is why the used 3090 stays the value pick for this tier.
What is the best local LLM for a 24GB GPU in 2026?
Qwen3.6 27B — ~17GB at Q4_K_M, ~30 tok/s on an RTX 4090, and it out-scores far larger models on agentic coding (68.9% SWE-bench Verified). If your work is pure code, Qwen 2.5 Coder 32B; for hard reasoning, DeepSeek-R1 32B.
Can 24GB VRAM run llama3.3:70b?
Not usably — a 70B needs ~40GB at Q4, so Ollama offloads heavily to system RAM and crawls. The classic budget route to 70B is 2×RTX 3090 (48GB). On a single 24GB card, the 27–32B class is the honest ceiling.
Related guides
Ready to Go Beyond Tutorials?
20 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?