Best Ollama Models 2026: 15 Ranked (Coding, Reasoning, Chat)
Want to go deeper than this article?
Free account unlocks the first chapter of all 22 courses — RAG, agents, MCP, voice AI, MLOps, real GitHub repos.
Ollama’s running. Here’s what to build with it. Go from “ollama run” to RAG apps, agents, and fine-tuned models — structured and hands-on. First chapter free.
The best Ollama models in 2026 are Qwen3.6-27B for overall quality on a single GPU (68.9% SWE-bench Verified, ~17GB VRAM), qwen3-coder:30b for coding on 24GB cards (256K context at small-model speed), and gpt-oss:20b as the pick for 16GB cards. DeepSeek R1 is still the reasoning model to beat, and for 8GB setups Llama 3.1 8B remains the most versatile option at ~5GB VRAM.
Quick Pick: Best Ollama Model for Your Hardware
| Your Setup | Best Model | Install Command |
|---|---|---|
| 8GB RAM, no GPU | Llama 3.2 3B | ollama pull llama3.2 |
| 16GB RAM / 8GB VRAM | Llama 3.1 8B | ollama pull llama3.1:8b |
| 16GB VRAM | gpt-oss:20b | ollama pull gpt-oss:20b |
| 24GB VRAM | Qwen3.6 27B | ollama pull qwen3.6:27b |
| 48GB+ VRAM | Llama 3.3 70B | ollama pull llama3.3:70b |
Why Ollama Model Choice Matters
Picking the right Ollama model is the single biggest factor in your local AI experience. A well-matched model runs fast, produces quality output, and fits your hardware. A poor choice gives you either slow responses or disappointing quality.
The Ollama library contains 500+ models, but most users only need to know about 10-15 models that consistently outperform the rest. This guide ranks those top models by task — coding, chat, reasoning, creative writing, and RAG — with real VRAM requirements and speed benchmarks so you can pick the right one immediately.
All models listed here are free, open-weight, and run entirely on your hardware. No API keys, no subscriptions, no data leaving your machine.
Reading articles is good. Building is better.
Free account = 20+ free chapters across 22 courses, with a per-chapter AI tutor. No card. Cancel anytime if you ever upgrade.
Top 15 Ollama Models Ranked
Overall Ranking (updated August 2026)
| Rank | Model | Parameters | VRAM (Q4) | Best For | HumanEval | MMLU | Speed (RTX 4090) |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.6 27B | 27B (dense) | ~17GB | Best overall on one GPU — 68.9% SWE-bench Verified | — | — | ~30 tok/s |
| 2 | Qwen3-Coder 30B | 30B MoE (~3B active) | ~18GB | Agentic coding, 256K context | — | — | ~50 tok/s |
| 3 | Qwen 2.5 Coder 32B | 32B | ~20GB | Strongest dense coder | 92.7% | 76.4 | ~35 tok/s |
| 4 | Llama 3.3 70B | 70B | ~40GB | General, 48GB+ rigs | 81.7% | 86.0 | ~18 tok/s |
| 5 | DeepSeek R1 32B | 32B | ~20GB | Reasoning, math | 72.6% | 79.8 | ~30 tok/s |
| 6 | gpt-oss:20b | 21B MoE (~3.6B active) | ~12-16GB | Reasoning + agentic on 16GB cards | — | — | Fast (MoE) |
| 7 | Qwen 2.5 32B | 32B | ~20GB | General, multilingual | 79.5% | 83.2 | ~35 tok/s |
| 8 | Llama 3.1 8B | 8B | ~5GB | General (budget) | 72.6% | 68.4 | ~85 tok/s |
| 9 | Qwen 2.5 Coder 7B | 7B | ~5GB | Coding (budget) | 88.4% | 64.2 | ~90 tok/s |
| 10 | DeepSeek R1 14B | 14B | ~9GB | Reasoning (mid-range) | 68.3% | 73.1 | ~55 tok/s |
| 11 | Gemma 4 12B Unified | 12B | ~7-8GB | Text + image + audio in one model, 256K context | — | — | — |
| 12 | Phi-4 Mini 3.8B | 3.8B | ~3GB | Small model king | 67.8% | 68.5 | ~110 tok/s |
| 13 | Qwen 2.5 Coder 1.5B | 1.5B | ~1.5GB | Autocomplete | 70.6% | 46.8 | ~150 tok/s |
| 14 | Nomic Embed Text | 137M | ~0.5GB | Embeddings, RAG | N/A | N/A | ~1000 tok/s |
| 15 | Llama 3.2 Vision 11B | 11B | ~8GB | Image understanding | N/A | 73.2 | ~50 tok/s |
Benchmark sources: HumanEval and MMLU scores from official model cards on Hugging Face and release announcements (Meta, Alibaba, DeepSeek, Google, Microsoft, OpenAI). The 2026-generation models (Qwen3.6, Qwen3-Coder, gpt-oss, Gemma 4) publish SWE-bench and agentic scores rather than classic HumanEval/MMLU, so those cells show a dash — their signature numbers are in the Best For column and the sections below. Speed figures are community-reported estimates from r/LocalLLaMA and vary by prompt length, context size, and system configuration. Your results may differ.
What moved in this update: Qwen3.6-27B takes #1 from Llama 3.3 70B because it needs less than half the VRAM and beats it on agentic coding, which makes it the better recommendation on the hardware most readers actually own. Qwen3-Coder 30B and gpt-oss:20b enter the top tier, and Google's Gemma 4 12B Unified replaces Gemma 2 27B. Mistral 7B, Mistral Small 24B, and Llama 3.2 3B drop out to make room — all three still work fine, they are just no longer top-15 picks.
What Are the Newest Ollama Models Worth Running in 2026?
The pace of new local models has not slowed. Since this guide first published, several releases have shifted what is worth pulling — especially the new wave of Mixture-of-Experts (MoE) models that activate only a few billion parameters per token, so they punch far above their size-on-disk while staying fast.
| Model | Total / Active Params | VRAM (Q4) | Best For | Notable Benchmark | Install |
|---|---|---|---|---|---|
| qwen3.6:27b | 27B dense | ~17GB | One-model coding + general use | 68.9% SWE-bench Verified — beats Qwen's own 397B MoE | ollama pull qwen3.6:27b |
| qwen3-coder:30b | 30B / ~3B (MoE) | ~18GB | Agentic coding, large repos | Strong SWE-bench (agentic), 256K native context | ollama pull qwen3-coder:30b |
| Devstral Small | 24B | ~15GB | Multi-file edits, agentic dev | ~46.8% SWE-bench Verified (2505 release) | ollama pull devstral:24b |
| gpt-oss:20b | 20.9B / ~3.6B (MoE) | ~12-16GB | Reasoning, agentic, general | OpenAI's open-weight model; runs on 16GB cards | ollama pull gpt-oss:20b |
| gpt-oss:120b | 116.8B / ~5.1B (MoE) | ~65GB+ | Frontier-class local reasoning | Fits a single 80GB GPU; strong agentic use | ollama pull gpt-oss:120b |
| DeepSeek R1 14B | 14B | ~9GB | Reasoning on mid hardware | Visible chain-of-thought | ollama pull deepseek-r1:14b |
What changed and what to pull:
- The dense surprise: Qwen3.6-27B. Alibaba's April 2026 dense 27B beats its own 397B MoE flagship on agentic coding (68.9% vs 65.4% SWE-bench Verified) while fitting a single RTX 4090/5090 at ~17GB Q4. If you have one good GPU and want one model for coding plus general work, this is the current default — full breakdown in our Qwen3.6-27B review.
- MoE is the headline.
qwen3-coder:30b(the "30B-A3B" build) carries 30B total parameters but activates only ~3B per token, so it generates at roughly 7B-class speed while reasoning closer to a 30B dense model. It is the new sweet spot for agentic coding on a 24GB GPU, and natively handles 256K tokens of context for repository-scale work. - OpenAI's gpt-oss landed on Ollama.
gpt-oss:20bruns on a 16GB card and is a genuine alternative to Qwen 2.5 32B for reasoning + agentic tasks;gpt-oss:120bis for workstation-class setups (single 80GB GPU or a 64GB+ Mac with offload). - Devstral is purpose-built for the read-edit-coordinate loop across multiple files, making it a better day-to-day agentic coder than a general 24B chat model. If your VRAM tops out at 24GB, run the largest coding-specialised model you can fit rather than a general one — the same principle our model size picker tool applies automatically.
Dense Qwen 2.5 Coder 32B and Llama 3.3 70B from the table above remain excellent and very widely deployed; the MoE arrivals don't retire them, they add a faster middle lane. For a deeper install-and-tune walkthrough of any of these, see the complete Ollama guide. And if your current GPU can't fit the model you want, the local AI hardware guide shows exactly what each budget tier unlocks.
Not sure which model your card can actually run? We ranked the best Ollama models by VRAM tier — 8GB, 12GB, 16GB, and 24GB — each with real Q4 footprints and tok/s. And the standout new small-footprint pick is Google's Gemma 4, which runs offline from a Raspberry Pi up to a single GPU.
Best Ollama Model for Coding
On a 24GB GPU, pull qwen3-coder:30b — it is the best Ollama coding model right now (~18GB at Q4, 256K native context, MoE so it generates at small-model speed). On an 8GB card, use Qwen 2.5 Coder 7B, and for editor autocomplete use Qwen 2.5 Coder 1.5B. If you want a single model for coding plus general work, Qwen3.6-27B (68.9% SWE-bench Verified, ~17GB) is the stronger all-rounder.
| Model | Size | VRAM | Benchmark / Strength | Install |
|---|---|---|---|---|
| Qwen3-Coder 30B | 30B MoE (~3B active) | ~18GB | Agentic coding, 256K native context | ollama pull qwen3-coder:30b |
| Qwen3.6 27B | 27B (dense) | ~17GB | 68.9% SWE-bench Verified; doubles as general model | ollama pull qwen3.6:27b |
| Qwen 2.5 Coder 32B | 32B | ~20GB | 92.7% HumanEval | ollama pull qwen2.5-coder:32b |
| Qwen 2.5 Coder 7B | 7B | ~5GB | 88.4% HumanEval | ollama pull qwen2.5-coder:7b |
| DeepSeek Coder V2 Lite | 16B | ~10GB | 81.1% HumanEval | ollama pull deepseek-coder-v2:16b |
| Qwen 2.5 Coder 1.5B | 1.5B | ~1.5GB | 70.6% HumanEval | ollama pull qwen2.5-coder:1.5b |
Why Qwen owns local coding: The Qwen 2.5 Coder series was trained on 5.5 trillion tokens of code data spanning 92 programming languages — the 32B scores 92.7% on HumanEval and the 7B at 88.4% outperforms models 4x its size. The newer Qwen3 generation adds MoE on top: qwen3-coder:30b activates only ~3B of its 30B parameters per token, so you get 30B-class code reasoning at roughly 7B-class generation speed, plus a 256K context that fits whole repositories.
Best setup for AI-assisted coding:
- Use Qwen 2.5 Coder 1.5B for fast autocomplete in Continue.dev or Cursor
- Use qwen3-coder:30b (24GB) or Qwen 2.5 Coder 7B (8GB) for chat, refactoring, and code review
- See our best local AI coding models guide, or the coding picks by VRAM tier: 8GB and 16GB
Reading articles is good. Building is better.
Free account = 20+ free chapters across 22 courses, with a per-chapter AI tutor. No card. Cancel anytime if you ever upgrade.
Best Models by Task
Best for General Chat
Chat models handle conversation, Q&A, summarization, and everyday tasks.
| Model | Size | VRAM | MMLU | Install |
|---|---|---|---|---|
| Llama 3.3 70B | 70B | ~40GB | 86.0 | ollama pull llama3.3:70b |
| Qwen 2.5 32B | 32B | ~20GB | 83.2 | ollama pull qwen2.5:32b |
| Mistral Small 24B | 24B | ~15GB | 81.0 | ollama pull mistral-small:24b |
| Llama 3.1 8B | 8B | ~5GB | 68.4 | ollama pull llama3.1:8b |
| Phi-4 Mini 3.8B | 3.8B | ~3GB | 68.5 | ollama pull phi4-mini |
Llama 3.3 70B is the best local chat model if you have the hardware. It replaced Llama 3.1 70B with better instruction following and reduced hallucination. For most users, Qwen 2.5 32B hits the sweet spot of quality and resource requirements — though if you only have one 24GB card, Qwen3.6-27B from the ranking above doubles as a strong chat model, and many single-GPU users now run it for everything.
Phi-4 Mini is remarkable at 3.8B parameters — it matches Llama 3.1 8B on MMLU while using 40% less VRAM.
Best for Reasoning and Math
Reasoning models excel at logic puzzles, math, analysis, and multi-step problem solving.
| Model | Size | VRAM | MATH | Install |
|---|---|---|---|---|
| DeepSeek R1 32B | 32B | ~20GB | 79.8 | ollama pull deepseek-r1:32b |
| DeepSeek R1 14B | 14B | ~9GB | 73.1 | ollama pull deepseek-r1:14b |
| DeepSeek R1 7B | 7B | ~5GB | 62.4 | ollama pull deepseek-r1:7b |
| Qwen 2.5 32B | 32B | ~20GB | 68.9 | ollama pull qwen2.5:32b |
DeepSeek R1 uses chain-of-thought reasoning — you can see the model's thinking process before it gives the final answer. This makes it significantly better at math, logic, and complex analysis compared to standard models. The 14B version is the best value: strong reasoning at just 9GB VRAM.
Best for RAG (Document Chat)
RAG models work alongside embedding models to answer questions from your documents.
For the language model (answers questions):
| Model | Size | VRAM | Why |
|---|---|---|---|
| Llama 3.1 8B | 8B | ~5GB | Best at grounding answers in provided context |
| Qwen 2.5 32B | 32B | ~20GB | Better comprehension for complex documents |
For the embedding model (indexes documents):
| Model | Size | VRAM | Install |
|---|---|---|---|
| nomic-embed-text | 137M | ~0.5GB | ollama pull nomic-embed-text |
| mxbai-embed-large | 335M | ~0.7GB | ollama pull mxbai-embed-large |
nomic-embed-text is the standard choice for RAG with Ollama. It works with Open WebUI, AnythingLLM, and most RAG frameworks. See our RAG local setup guide for a complete walkthrough.
Best for Vision (Image Understanding)
| Model | Size | VRAM | Install |
|---|---|---|---|
| Llama 3.2 Vision 11B | 11B | ~8GB | ollama pull llama3.2-vision:11b |
| Llama 3.2 Vision 90B | 90B | ~55GB | ollama pull llama3.2-vision:90b |
Vision models can describe images, read text from screenshots (OCR), analyze charts, and answer questions about photos. The 11B version handles most tasks well at 8GB VRAM.
Uncensored and Low-Refusal Models
The most popular low-refusal models on Ollama are the Dolphin fine-tunes: dolphin3 (Llama 3.1 8B base), dolphin-mistral (7B), and dolphin-mixtral (8x7B MoE). They are community retrains that strip most of the refusal behavior out of the base model, so they will answer questions the stock models decline — useful for fiction writers working with dark themes, security researchers, and anyone tired of refusals on harmless prompts.
Two honest caveats. First, the trade-off is real: the Dolphin models ride on older bases (Llama 3.1, Mistral 7B, Mixtral), so raw capability sits a clear generation behind Qwen3.6 or Llama 3.3 — you give up quality for fewer refusals. Second, "uncensored" removes the guardrails, not your responsibility: these models will confidently produce wrong or harmful text with no pushback, and what you generate and publish is on you. Run them for the use cases that need them, not as your daily driver.
Install with ollama pull dolphin3, ollama pull dolphin-mistral, or ollama pull dolphin-mixtral:8x7b. We cover the Mixtral variant in more depth in our Dolphin Mixtral 8x7B review, and if you are after the image-generation side of this topic, see uncensored local image generation. For the full text-model guide — abliterated builds, VRAM needs per card, and honest quality trade-offs — see Best Uncensored Local LLMs.
The Ollama Library at a Glance
Ollama can run any model in its public library — hundreds of open-weight chat, coding, reasoning, vision, and embedding models — plus any GGUF file you import yourself. The top-15 above covers our ranked picks; the table below is the wider map of what people actually pull, with the sizes on offer and the VRAM you need at the default Q4 quantization.
| Model | Category | Sizes | VRAM (Q4, bold size) | Pull command |
|---|---|---|---|---|
| qwen3.6 | Coding + general | 27b | ~17GB | ollama pull qwen3.6:27b |
| llama3.3 | General chat | 70b | ~40GB | ollama pull llama3.3:70b |
| llama3.1 | General chat | 8b, 70b, 405b | ~5GB | ollama pull llama3.1:8b |
| llama3.2 | Small / edge | 1b, 3b | ~2GB | ollama pull llama3.2 |
| llama4 | MoE general | scout (109b), maverick (400b) | ~65GB+ | ollama pull llama4:scout |
| qwen3 | General, hybrid reasoning | 0.6b–235b, 14b | ~9GB | ollama pull qwen3:14b |
| qwen2.5 | General, multilingual | 0.5b–72b, 32b | ~20GB | ollama pull qwen2.5:32b |
| gemma4 | Multimodal general | e2b, e4b, 12b, 26b, 31b | ~7-8GB | ollama pull gemma4:12b |
| gemma3 | Multimodal general | 1b, 4b, 12b, 27b | ~8GB | ollama pull gemma3:12b |
| gpt-oss | Reasoning, agentic | 20b, 120b | ~12-16GB | ollama pull gpt-oss:20b |
| phi4 | STEM / reasoning | 14b | ~9GB | ollama pull phi4 |
| phi4-mini | Small general | 3.8b | ~3GB | ollama pull phi4-mini |
| mistral | General | 7b | ~4.5GB | ollama pull mistral |
| mistral-small | General, multilingual | 24b | ~15GB | ollama pull mistral-small:24b |
| mistral-nemo | General, 128K context | 12b | ~7GB | ollama pull mistral-nemo |
| mixtral | MoE general | 8x7b, 8x22b | ~26GB | ollama pull mixtral:8x7b |
| command-r | RAG, tool use | 35b | ~20GB | ollama pull command-r |
| granite3.3 | Enterprise tasks | 2b, 8b | ~5GB | ollama pull granite3.3 |
| smollm2 | Tiny / edge | 135m–1.7b | ~1GB | ollama pull smollm2 |
| tinyllama | Tiny / edge | 1.1b | ~1GB | ollama pull tinyllama |
| qwen3-coder | Agentic coding | 30b (MoE) | ~18GB | ollama pull qwen3-coder:30b |
| qwen2.5-coder | Coding | 0.5b–32b | ~20GB | ollama pull qwen2.5-coder:32b |
| devstral | Agentic coding | 24b | ~15GB | ollama pull devstral:24b |
| deepseek-coder-v2 | Coding (MoE) | 16b, 236b | ~10GB | ollama pull deepseek-coder-v2:16b |
| codegemma | Coding | 2b, 7b | ~5GB | ollama pull codegemma |
| starcoder2 | Code completion | 3b, 7b, 15b | ~5GB | ollama pull starcoder2:7b |
| codellama | Coding (legacy) | 7b–70b | ~5GB | ollama pull codellama |
| deepseek-r1 | Chain-of-thought | 1.5b–671b, 14b | ~9GB | ollama pull deepseek-r1:14b |
| qwq | Chain-of-thought | 32b | ~20GB | ollama pull qwq |
| magistral | Reasoning | 24b | ~15GB | ollama pull magistral |
| llama3.2-vision | Vision | 11b, 90b | ~8GB | ollama pull llama3.2-vision:11b |
| qwen2.5vl | Vision, OCR | 3b, 7b, 32b, 72b | ~5GB | ollama pull qwen2.5vl |
| llava | Vision | 7b, 13b, 34b | ~5GB | ollama pull llava |
| minicpm-v | Vision | 8b | ~5GB | ollama pull minicpm-v |
| moondream | Tiny vision | 1.8b | ~2GB | ollama pull moondream |
| nomic-embed-text | Embeddings | 137m | ~0.5GB | ollama pull nomic-embed-text |
| mxbai-embed-large | Embeddings | 335m | ~0.7GB | ollama pull mxbai-embed-large |
| bge-m3 | Embeddings, multilingual | 567m | ~1GB | ollama pull bge-m3 |
| all-minilm | Embeddings (light) | 23m, 33m | <0.5GB | ollama pull all-minilm |
| dolphin3 | Low-refusal | 8b | ~5GB | ollama pull dolphin3 |
| dolphin-mistral | Low-refusal | 7b | ~4.5GB | ollama pull dolphin-mistral |
VRAM figures follow the same rule of thumb as the rest of this guide (~0.6GB per billion parameters at Q4, plus overhead) and refer to the bolded size where a family ships several. Ollama also runs any custom GGUF via a Modelfile FROM line — the complete Ollama guide shows how. To size a specific model against your card, use the VRAM calculator.
Models by Hardware Budget
8GB RAM / No Dedicated GPU
You're limited to 3B-4B parameter models with CPU inference. Expect 5-15 tok/s.
# Best picks for 8GB RAM
ollama pull llama3.2 # 3B - best general quality
ollama pull phi4-mini # 3.8B - surprisingly capable
ollama pull gemma2:2b # 2B - fastest, basic tasks
16GB RAM / 8GB VRAM (RTX 3060, M1/M2 16GB)
The sweet spot for most users. 7B-8B models run at full GPU speed.
# Best picks for 16GB / 8GB VRAM
ollama pull llama3.1:8b # Best general-purpose 8B
ollama pull qwen2.5-coder:7b # Best coding 7B
ollama pull deepseek-r1:7b # Reasoning with chain-of-thought
ollama pull nomic-embed-text # Embeddings for RAG
16GB VRAM (RTX 4080, RTX 4060 Ti 16GB)
The new MoE wave made 16GB genuinely useful — 20B-class quality without a flagship card.
# Best picks for 16GB VRAM
ollama pull gpt-oss:20b # OpenAI open-weight MoE — the 16GB headliner
ollama pull qwen3:14b # Best dense 14B all-rounder
ollama pull gemma4:12b # Text + image + audio in one model
ollama pull deepseek-r1:14b # Reasoning with chain-of-thought
24GB VRAM (RTX 4090, M3 Pro 36GB)
Access to the 27B-32B class — a massive quality jump over 8B.
# Best picks for 24GB VRAM
ollama pull qwen3.6:27b # Best overall — coding + general (68.9% SWE-bench)
ollama pull qwen3-coder:30b # Agentic coding, 256K context, MoE speed
ollama pull qwen2.5-coder:32b # Strongest dense coder (92.7% HumanEval)
ollama pull deepseek-r1:32b # Best reasoning model
48GB+ VRAM (RTX 5090 32GB + offload, 2x GPUs, M4 Max 64GB)
Run 70B models — comparable to GPT-4 turbo.
# Best picks for 48GB+
ollama pull llama3.3:70b # Best overall local model
ollama pull qwen2.5:72b # Excellent multilingual
Speed and VRAM Reference Table
| Model | Q4_K_M Size | VRAM Used | RTX 3060 12GB | RTX 4090 24GB | Mac M4 Max 64GB |
|---|---|---|---|---|---|
| Gemma 2 2B | 1.6 GB | ~2 GB | 110 tok/s | 150 tok/s | 80 tok/s |
| Llama 3.2 3B | 2.0 GB | ~3 GB | 90 tok/s | 120 tok/s | 65 tok/s |
| Phi-4 Mini 3.8B | 2.5 GB | ~3 GB | 85 tok/s | 110 tok/s | 60 tok/s |
| Mistral 7B | 4.1 GB | ~5 GB | 55 tok/s | 90 tok/s | 40 tok/s |
| Llama 3.1 8B | 4.7 GB | ~6 GB | 50 tok/s | 85 tok/s | 38 tok/s |
| Qwen 2.5 Coder 7B | 4.4 GB | ~5 GB | 55 tok/s | 90 tok/s | 40 tok/s |
| DeepSeek R1 14B | 8.7 GB | ~10 GB | 25 tok/s | 55 tok/s | 28 tok/s |
| Mistral Small 24B | 14 GB | ~16 GB | CPU only | 40 tok/s | 25 tok/s |
| Gemma 2 27B | 16 GB | ~18 GB | CPU only | 38 tok/s | 24 tok/s |
| Qwen 2.5 32B | 19 GB | ~21 GB | CPU only | 35 tok/s | 22 tok/s |
| DeepSeek R1 32B | 19 GB | ~21 GB | CPU only | 30 tok/s | 20 tok/s |
| Llama 3.3 70B | 40 GB | ~42 GB | CPU only | CPU offload | 18 tok/s |
Speeds are community-reported estimates and vary significantly by prompt length, context size, quantization, and system load. Treat as rough comparisons, not precise measurements.
How to Pick the Right Model
Decision Flowchart
Step 1: What's your VRAM?
- Under 4GB → Gemma 2 2B or Llama 3.2 1B
- 4-8GB → 7B-8B models
- 8-16GB → 14B-24B models
- 16-24GB → 32B models
- 24GB+ → 70B models
Step 2: What's your primary use case?
- General chat → Qwen3.6-27B on a 24GB card, else Llama 3.1/3.3 or Qwen 2.5 (largest that fits)
- Coding → qwen3-coder:30b on 24GB, else Qwen 2.5 Coder (largest that fits)
- Reasoning/math → DeepSeek R1 (largest that fits)
- Fast autocomplete → Qwen 2.5 Coder 1.5B
- Document Q&A → Llama 3.1 8B + nomic-embed-text
Step 3: Speed vs Quality?
- Need fast responses → Pick one size down from your maximum
- Need best quality → Pick the largest that fits your VRAM
- Running multiple models → Leave 4-6GB headroom for the OS and second model
Common Mistakes to Avoid
- Running a model that barely fits — If your model uses 23.5GB of 24GB VRAM, you'll get swapping and slowdowns. Leave 2-3GB headroom.
- Using general models for coding — Qwen 2.5 Coder 7B massively outperforms Llama 3.1 8B on code tasks despite being smaller. Use specialized models.
- Ignoring quantization — Always use Q4_K_M (Ollama default). Full precision wastes VRAM with negligible quality gain.
- Chasing parameter count — A well-trained 32B model (Qwen 2.5) often outperforms a mediocre 70B model. Quality of training data matters more than size alone.
Model Management Tips
Check Installed Models
ollama list
# NAME ID SIZE MODIFIED
# llama3.1:8b 365c0bd3c000 4.7 GB 2 days ago
# qwen2.5-coder:7b 12345abc 4.4 GB 1 day ago
Free Up Disk Space
# Remove models you no longer use
ollama rm codellama:7b
ollama rm mistral:7b
# Models are stored in:
# macOS: ~/.ollama/models
# Linux: /usr/share/ollama/.ollama/models
# Windows: C:\Users\<user>\.ollama\models
Pull Specific Quantizations
# Default (Q4_K_M) — best balance
ollama pull llama3.1:8b
# Higher quality (Q5_K_M) — 10-15% more VRAM
ollama pull llama3.1:8b-instruct-q5_K_M
# Smallest (Q2_K) — 30% less VRAM, noticeable quality loss
ollama pull llama3.1:8b-instruct-q2_K
Set Context Window Size
# In Ollama chat, increase context window:
/set parameter num_ctx 8192
# Or create a Modelfile for persistent settings:
# Create a file called Modelfile:
# FROM llama3.1:8b
# PARAMETER num_ctx 8192
# PARAMETER temperature 0.7
# Then: ollama create my-llama -f Modelfile
Does the Model Runner Change Which Model Is "Best"?
A question that comes up constantly: do these rankings change if you run models through LM Studio or Msty instead of Ollama? The short answer is no — the model weights are identical. A Q4_K_M GGUF of Qwen 2.5 Coder 32B produces the same quality whether Ollama, LM Studio, or Msty loads it. What differs is the experience around the model.
- Ollama is the lightest, most scriptable runner — a single
ollama pull/ollama runworkflow and a built-in API that every tool in this guide targets. Best if you want models available to other apps. - LM Studio adds a polished GUI, a model browser, and easy quantization switching — friendlier if you prefer clicking over the terminal.
- Msty layers chat features (split chats, knowledge stacks, web search) on top.
If you're deciding which runner to standardise on before you commit to a model library, our Msty vs Ollama vs LM Studio comparison breaks down the trade-offs in detail. The practical takeaway: pick your models from this guide first, then pick the runner that matches how you like to work — they all read the same GGUF files.
One real-world caveat: VRAM headroom and quantization defaults can differ slightly between runners, which nudges the largest model you can comfortably load. If you're on a tight 8GB or 12GB budget, our best local AI models for 8GB RAM guide lists the picks that stay fast regardless of which runner you choose.
Key Takeaways
- Qwen3.6-27B is the best single-GPU model — 68.9% SWE-bench Verified at ~17GB
- qwen3-coder:30b is the best coding pick for 24GB cards; Qwen 2.5 Coder 32B is still the dense HumanEval leader (92.7%)
- Llama 3.3 70B is the best overall model if you have 48GB+ VRAM/RAM
- DeepSeek R1 is the best reasoning model with visible chain-of-thought
- Phi-4 Mini 3.8B punches far above its weight for small hardware
- Always use Q4_K_M quantization (Ollama default) — best quality-per-VRAM
- Match model to task — specialized models (Coder, R1) beat general models on their domains
- nomic-embed-text is the go-to embedding model for RAG
Next Steps
- Set up Open WebUI for a ChatGPT-like interface with your models
- Find models for 8GB RAM if you're on limited hardware
- Set up Continue.dev for AI coding with Ollama
- Compare Jan vs LM Studio vs Ollama for model management
- Check VRAM requirements for detailed GPU sizing
- Run GPT-OSS locally — OpenAI's first open-source model on Ollama
- Run Llama 4 Scout locally — Meta's 109B MoE with 10M token context
- Try Qwen3-Coder — Alibaba's best coding model (70.6% SWE-bench)
- RTX 5090 vs 5080 for local AI — which GPU to buy for running models
- LMArena leaderboard explained — how AI models are ranked by 6M+ votes
The Ollama model ecosystem evolves rapidly. We test and update this ranking monthly. Last verified August 2026.
Ollama’s running. Here’s what to build with it.
Go from “ollama run” to RAG apps, agents, and fine-tuned models — structured and hands-on. First chapter free.
Liked this? 20 full AI courses are waiting.
From fundamentals to RAG, agents, MCP servers, voice AI, and production deployment with real GitHub repos. First chapter free, every course.
Build Real AI on Your Machine
RAG, agents, NLP, vision, and MLOps - chapters across 22 courses that take you from reading about AI to building AI.
Want structured AI education?
22 courses, 519+ chapters, from $9. Understand AI, don't just use it.
Continue Your Local AI Journey
- 15 Best Free AI Models to Run Locally with Ollama (2026) — No API Key
- Best Local LLMs for Tool & Function Calling (2026 Tested)
- Best Ollama Models for 8GB RAM 2026: 12 Tested Local Picks
- Best Ollama Models for AI Agents 2026: 9 Tested & Ranked
- Best Uncensored Local LLMs: Abliterated Ollama Models
- Build a Local AI Slack & Discord Bot with Ollama (Full Tutorial)
- Build a Local RAG Pipeline: Ollama + ChromaDB Step-by-Step
- Build a Telegram Bot with Local AI (Ollama + Python Tutorial)
- CodeLlama Instruct 7B: Ollama Setup, HumanEval (2026)
- Complete Ollama Guide 2026: Install, Run & Manage 500+ Local AI Models
Comments (0)
No comments yet. Be the first to share your thoughts!