Best Ollama Models for 12GB VRAM (12GB VRAM)
RTX 4070 / RTX 3060 12GB — the tier the 14B class was made for
The best Ollama model for 12GB VRAM in 2026 is Phi-4 14B — ~9GB at Q4_K_M, with a bandwidth ceiling of ~40 tok/s on an RTX 3060 12GB and ~55 on an RTX 4070 (360 and 504 GB/s ÷ 9.1GB — ceilings, not measurements). 12GB is the 14B sweet spot: Qwen 3 14B and DeepSeek-R1 14B fit too, with Gemma 3 12B for multilingual work and Llama 3.1 8B as the fast lane.
Models that fit in 12GB VRAM
Reference card: RTX 4070 / RTX 3060 12GB. Figures are for Q4_K_M with a modest context window.
| Model | Size | Build | VRAM used | Ceiling tok/s* |
|---|---|---|---|---|
| WINNERPhi-4 14B The 14B that punches hardest for its size — STEM/reasoning standout, with ~3GB left for context on 12GB. ollama pull phi4 | 14B | Q4_K_M | ~9.1GB | ceiling ~40 tok/s (3060: 360 ÷ 9.1GB) · ~55 (4070: 504 ÷ 9.1GB) |
| Qwen 3 14B Best all-rounder of the 14B class (it takes the crown at 16GB, where its long context fits too). ollama pull qwen3:14b | 14B | Q4_K_M | ~9.3GB | ceiling ~39 tok/s (3060: 360 ÷ 9.3GB) · ~54 (4070: 504 ÷ 9.3GB) |
| DeepSeek-R1 14B Visible chain-of-thought — the best hard-problems pick that fits 12GB. ollama pull deepseek-r1:14b | 14B | Q4_K_M | ~9.0GB | ceiling ~40 tok/s (3060: 360 ÷ 9.0GB) · ~56 (4070: 504 ÷ 9.0GB) |
| Gemma 3 12B Best multilingual and writing quality, with a bit more KV-cache headroom than the 14Bs. ollama pull gemma3:12b | 12B | Q4_K_M | ~8.2GB | ceiling ~44 tok/s (3060: 360 ÷ 8.2GB) · ~61 (4070: 504 ÷ 8.2GB) |
| Llama 3.1 8B The speed pick — leaves ~7GB free, so a long context or a second small model fits alongside. ollama pull llama3.1:8b | 8B | Q4_K_M | ~5.0GB | ceiling ~73 tok/s (3060: 360 ÷ 4.9GB) · ~103 (4070: 504 ÷ 4.9GB) |
* Ceiling tok/s = the card’s vendor-listed memory bandwidth ÷ the model’s file size as Ollama lists it — the rate a dense model cannot exceed, because every generated token reads the whole weight set once. We have not measured these cards. Real-world generation usually lands at 50–80% of the ceiling and falls as the context grows; mixture-of-experts rows read only their active parameters, so no ceiling is quoted for them.
What won't fit in 12GB
- ✗Codestral 22B (needs ~13GB) — Needs 16GB — on 12GB stick to Qwen 2.5 Coder 14B (~9GB) for code.
- ✗gpt-oss:20b (needs ~14–16GB) — OpenAI’s open-weight MoE is a hair too big — it is the headline pick one tier up.
How to fit more in 12GB
- →12GB exists for the 14B class: ~9GB of weights plus 2–3GB of KV cache is exactly the budget. Stay at Q4_K_M — a 14B at Q5 (~11GB) leaves no room for context here.
- →Keep num_ctx ≤8K with a 14B loaded; switch to the 8B when a task genuinely needs a long context on 12GB.
- →A used RTX 3060 12GB is the cheapest ticket into this tier — the RTX 4070 runs the same models with a 40% higher bandwidth ceiling (504 vs 360 GB/s).
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run phi4Go from "it runs" to actually building
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
What is the best Ollama model for an RTX 3060 12GB?
Phi-4 14B (~9GB at Q4, bandwidth ceiling ~40 tok/s on the 3060) for reasoning-heavy work, Qwen 3 14B as the all-rounder, and DeepSeek-R1 14B when you want visible chain-of-thought. All three are the same ~9GB size — pick by task, not by fit.
Can 12GB VRAM run a 14B model?
Yes — comfortably, and that is the whole point of this tier. A 14B at Q4_K_M uses ~9GB, leaving 2–3GB for the KV cache. It is the biggest dense class that fits; 20B+ models (Codestral, gpt-oss:20b) need 16GB.
RTX 4070 vs RTX 3060 12GB for Ollama?
Both have 12GB, so the model list is identical — the 4070’s bandwidth ceiling is 40% higher (504 vs 360 GB/s). If budget decides, the used 3060 12GB runs everything on this page; the 4070 just makes the 14Bs feel snappier.
Related guides
Ready to Go Beyond Tutorials?
25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?