Best Ollama Models for 6GB VRAM: 6 That Actually Fit (6GB VRAM)
GTX 1060, RTX 2060, RTX 3050, RTX 4050 laptop — the pull commands that stay on the GPU
The best Ollama model for 6GB VRAM in 2026 is qwen3.5:4b — 3.4GB in the Ollama library, with tool calling, thinking and image input, which leaves roughly 2GB of a 6GB card for context and the desktop. Below it, phi4-mini (2.5GB), granite4.1:3b (2.1GB), llama3.2:3b (2.0GB) and qwen2.5-coder:3b (1.9GB) all fit with room to spare, and gemma4:e2b-it-qat (4.3GB) is the newest Gemma that fits — but only via the QAT tag, because the default gemma4:e2b build is 7.2GB. The trap on a 6GB card is not the weights, it is the context window you set afterwards.
Models that fit in 6GB VRAM
Tested reference: GTX 1060 6GB / RTX 2060 / RTX 3050 6GB / RTX 4050 laptop. Figures are for Q4_K_M with a modest context window.
| Model | Size | Build | VRAM used | Speed |
|---|---|---|---|---|
| WINNERqwen3.5:4b The best-equipped model that fits: tools, thinking, image input and a 262K native context. Ollama’s 3.4GB is the 2.74 GB Q4_K_M weights plus the 0.67 GB vision projector, so text-only generation streams about 2.7GB per token. Qwen’s own reported table gives it 79.1 MMLU-Pro and 79.9 on TAU2-Bench, which is why it beats the older 3B class here rather than just being newer. ollama pull qwen3.5:4b | 4B (dense, vision) | Q4_K_M + vision projector | 3.4GB | mid · ~2.7GB/token |
| gemma4:e2b-it-qat The newest Gemma that fits a 6GB card, and only in this build — the default gemma4:e2b tag is the 7.2GB q4_K_M and will not load on-GPU. 4.3GB leaves under 1.5GB for everything else, so keep the context small and nothing else on the card. ollama pull gemma4:e2b-it-qat | E2B (MatFormer, vision) | int4 QAT | 4.3GB | slowest here · ~4.3GB/token |
| phi4-mini Microsoft’s small model, and the lightest way to a strong 6GB default. One caution the size hides: 32 layers × 2 × 8 KV heads × 128 head_dim × 2 bytes = 128 KiB per token, so an 8K window costs a full 1GB of KV cache — four times what qwen3.5:4b charges for the same window. ollama pull phi4-mini | 3.8B | Q4_K_M | 2.5GB | fast · ~2.5GB/token |
| granite4.1:3b IBM’s Apache-2.0 small model with tool calling and a 128K listed context. Its head_dim is 64 rather than 128, so KV works out at 40 × 2 × 8 × 64 × 2 = 80 KiB/token — cheaper per token of context than phi4-mini despite the deeper stack. ollama pull granite4.1:3b | 3B | Q4_K_M | 2.1GB | fast · ~2.1GB/token |
| llama3.2:3b The safe default when a tool, script or tutorial assumes a Llama. Nothing about it is new in 2026, but it is small, well documented and the fine-tune ecosystem around it is still the largest. ollama pull llama3.2:3b | 3B | Q4_K_M | 2.0GB | fast · ~2.0GB/token |
| qwen2.5-coder:3b The coding pull for this tier, and the smallest thing here. 36 layers × 2 × 2 KV heads × 128 head_dim × 2 bytes = 36 KiB/token, so even a 16K code context is only ~576MB. Pairs with qwen2.5-coder:1.5b (986MB) for autocomplete. ollama pull qwen2.5-coder:3b | 3B | Q4_K_M | 1.9GB | fastest · ~1.9GB/token |
What won't fit in 6GB
- ✗gemma4:e2b (default tag) (needs 7.2GB) — The same model as the QAT row above, in the q4_K_M build Ollama serves by default. Bigger than the whole card — this is the single most likely mis-pull at this tier.
- ✗qwen3.5:9b (needs 6.6GB) — The 9B of the same family, and 0.6GB more than a 6GB card has in total. It is the winner one tier up.
- ✗ornith:9b (needs 5.6GB) — The new agentic 9B. The weights alone would leave under 0.5GB for KV cache and the desktop, so Ollama will offload layers to the CPU and the speed collapses.
- ✗granite4.1:8b (needs 5.3GB) — Same story one notch down — technically loadable, practically not. granite4.1:3b is the 6GB Granite.
- ✗llama3.1:8b (needs 4.9GB) — The classic 8B ask, and the most common disappointment at this tier. It loads, then leaves roughly a gigabyte for the KV cache and the desktop to share. It is an 8GB model.
- ✗deepseek-r1:8b (needs 5.2GB) — Reasoning models emit long thinking traces, so they need the most context headroom, which is exactly what a 6GB card has least of. Use deepseek-r1:1.5b (1.1GB) if you want the visible chain-of-thought at this tier.
How to fit more in 6GB
- →The "speed" column is bytes read per generated token, not a benchmark. Decode is memory-bandwidth bound, so tok/s ≈ your card’s bandwidth ÷ that figure. A GTX 1060 and an RTX 4050 laptop run this identical list at very different rates, which is why no absolute tok/s is quoted here.
- →Load a model, then run `ollama ps` in a second terminal. The PROCESSOR column must read 100% GPU — anything with a CPU share means it spilled past your 6GB and you are now running at system-RAM speed. Ollama’s own docs use exactly this check.
- →Context, not weights, is what kills a 6GB card. Ollama defaults anything under 24 GiB of VRAM to a 4K context; that default is protecting you. Raising it with OLLAMA_CONTEXT_LENGTH=64000 ollama serve on a 6GB card is how a model that loaded fine yesterday starts offloading today.
- →If you need a longer window, buy it with KV quantisation rather than a smaller model: Ollama supports OLLAMA_KV_CACHE_TYPE=q8_0 with flash attention on, which the docs describe as roughly half the memory of f16 with very little quality loss.
- →Run one model at a time. Ollama keeps recent models resident and will happily load a second; on 6GB that is an instant spill. Set OLLAMA_KEEP_ALIVE=0 if you switch models often, or unload explicitly with a keep_alive of 0.
- →On a laptop, the desktop is already spending your VRAM. Windows plus a hardware-accelerated browser can hold 0.5–1GB before Ollama loads anything, which is the difference between the 4.3GB Gemma fitting and not.
- →gemma3:4b (3.3GB) is the sensible substitute if the Gemma 4 QAT build gives you trouble — same tier, same vendor, one generation older, and it has been on 6GB cards for a year.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run qwen3.5:4bollama ps # PROCESSOR must read 100% GPUOLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serveGo from "it runs" to actually building
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
What are the best Ollama models for 6GB VRAM?
qwen3.5:4b (3.4GB) as the all-rounder, phi4-mini (2.5GB) and granite4.1:3b (2.1GB) as lighter defaults, llama3.2:3b (2.0GB) when something expects a Llama, qwen2.5-coder:3b (1.9GB) for code, and gemma4:e2b-it-qat (4.3GB) if you want the newest Gemma. All six sizes are the ones Ollama publishes in its own library.
Can Ollama run an 8B model on 6GB VRAM?
It will start, and then it will disappoint you. llama3.1:8b is 4.9GB and deepseek-r1:8b is 5.2GB, so the weights nearly fill the card and the KV cache has nowhere to go — Ollama offloads layers to system RAM and generation drops to a crawl. Check with `ollama ps`: if PROCESSOR is not 100% GPU, that is what happened. The 8B class is an 8GB-card class.
Why did my model fit yesterday and not today?
Almost always the context window. Ollama defaults cards under 24 GiB of VRAM to a 4K context; raise it for an agent or a coding tool and the KV cache grows linearly with it. On phi4-mini that is 128 KiB per token, so going from 4K to 32K adds about 3.5GB — more than the model itself. Either drop the window or turn on flash attention with OLLAMA_KV_CACHE_TYPE=q8_0.
Is a 6GB GPU worth using for Ollama at all?
Yes, with the list narrowed. Six of the models above fit fully on-GPU with real context left over, including a multimodal 4B with tool calling and a dedicated code model. What you cannot do is run the 8B–14B models most "best Ollama models" lists assume, and no quantisation trick changes that on 6GB.
Related guides
Ready to Go Beyond Tutorials?
20 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?