Best LLM for 12GB VRAM (12GB VRAM)
The 14B class at Q4 — what a 12GB GPU is actually for
The best local LLM for 12GB VRAM in 2026 is Phi-4 14B — ~9GB at Q4_K_M and ~25–32 tokens/sec on an RTX 3060 12GB or RTX 4070. 12GB is the cheapest ticket into the 14B class: Qwen 3 14B (the all-rounder) and DeepSeek-R1 14B (visible reasoning) are the same ~9GB, so pick by task, not by fit. Gemma 3 12B trades a little size for extra context headroom. A 32B does not fit — that is what 24GB cards are for.
Models that fit in 12GB VRAM
Tested reference: RTX 3060 12GB / RTX 4070. Figures are for Q4_K_M with a modest context window.
| Model | Size | Build | VRAM used | Speed |
|---|---|---|---|---|
| WINNERPhi-4 14B Best overall at 12GB — punches far above its size on math and logic, with ~3GB left over for the KV cache. ollama pull phi4 | 14B | Q4_K_M | ~9.0GB | ~25–32 tok/s |
| Qwen 3 14B The all-rounder — coding, reasoning and a 128K context (cap num_ctx well below that on 12GB). ollama pull qwen3:14b | 14B | Q4_K_M | ~9.0GB | ~25–32 tok/s |
| DeepSeek-R1 14B Distilled reasoning with visible chain-of-thought — the hard-problems pick. ollama pull deepseek-r1:14b | 14B | Q4_K_M | ~9.0GB | ~25–30 tok/s |
| Gemma 3 12B Best multilingual and prose at this tier — and its smaller footprint buys ~1GB more context headroom than the 14Bs. ollama pull gemma3:12b | 12B | Q4_K_M | ~8.0GB | ~28–35 tok/s |
| Llama 3.1 8B The speed lane — leaves ~7GB free for a genuinely long context or a second small model. ollama pull llama3.1:8b | 8B | Q4_K_M | ~5.0GB | ~50–60 tok/s |
What won't fit in 12GB
- ✗Codestral 22B (needs ~13GB) — Needs 16GB; for code on 12GB run Qwen 2.5 Coder 14B instead.
- ✗gpt-oss:20b (needs ~14–16GB) — OpenAI’s open-weight MoE wants a 16GB card.
- ✗Qwen 2.5 32B (needs ~20GB) — Full-quality 32B needs 24GB; even the Q3 squeeze (~15GB) is a 16GB trick.
How to fit more in 12GB
- →All three 14Bs are the same ~9GB, so choose by task — Phi-4 for STEM, Qwen 3 for general work and coding, DeepSeek-R1 when you want to watch it reason. There is no “lighter” 14B to hunt for.
- →Turn on OLLAMA_FLASH_ATTENTION=1 before loading a 14B: the ~3GB left after weights goes much further and comfortably holds a 16K context.
- →A 12GB card can hold Llama 3.1 8B (~5GB) and a 3B (~2.2GB) resident at once — a chat model plus an instant-autocomplete model, something 8GB cannot do.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run phi4Go from "it runs" to actually building
All 519 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
Is 12GB VRAM enough for local AI?
Yes — 12GB is the mid-range sweet spot. The entire 14B class (Phi-4, Qwen 3 14B, DeepSeek-R1 14B) runs at Q4_K_M in ~9GB at 25–32 tok/s, a clear quality step over the 7–8B models an 8GB card caps out at.
Can 12GB VRAM run a 32B model?
No. Qwen 2.5 32B needs ~20GB at Q4 and still ~15GB at an aggressive Q3 — both past 12GB. Ollama would offload most layers to system RAM and crawl. The honest ceiling at 12GB is the 14B class.
Which GPUs have 12GB VRAM?
RTX 3060 12GB, RTX 4070, RTX 4070 Super and RTX 5070 all carry 12GB. They run the same model list — a used 3060 is the cheapest way in; the 4070-class cards just run the same 14Bs 30–40% faster.
Related guides
Ready to Go Beyond Tutorials?
20 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?