AI Model Recommender
Not sure which model to run? Select your task and hardware — we'll recommend local AI models that fit, with install commands, VRAM requirements, and the reason each one ranks where it does. Works for Ollama, llama.cpp, and any local inference engine.
Find Your Perfect Model
Top 5 Models for General Chat (24 GB (RTX 4090 / Mac 24GB))
Largest general model that fits a single 24GB card
General model; among those that fit, larger ranks higher.
ollama run qwen2.5:32bStrong general model for 12-16GB cards
General model; among those that fit, larger ranks higher.
ollama run qwen2.5:14bGoogle open model for general tasks
General model; among those that fit, larger ranks higher.
ollama run gemma2:9bBalanced 8B model with a 128K context window
General model; among those that fit, larger ranks higher.
ollama run llama3.1:8bReliable all-rounder at 7B
General model; among those that fit, larger ranks higher.
ollama run mistralUpgrade to unlock these models
Sizes at Q4_K_M quantization, from the Ollama registry. Speeds are bandwidth ceilings (memory bandwidth ÷ file size), not measurements. See our VRAM Calculator for exact memory requirements, or our GPU comparison for hardware recommendations.
How We Rank Models
The tool does not assign quality scores. Among the models tagged for your task that fit your memory, models built for that task rank first, then larger checkpoints. To check a model's quality claims, these are the public benchmarks to read, alongside the practical factors the tool does use:
Benchmark Sources
- Coding: HumanEval, SWE-bench Verified, Aider polyglot
- Reasoning: MATH, GPQA Diamond, ARC-AGI
- General: MMLU, LMArena ELO
- Creative: AlpacaEval, human preference ratings
Practical Factors
- VRAM efficiency: Q4_K_M quantization size from GGUF files
- Speed: a bandwidth ceiling on an RTX 4090 (1,008 GB/s ÷ file size), not a measurement
- Ecosystem: Ollama availability, community support, documentation
- License: All recommended models are free for commercial use
Key principle: A model that fits entirely in your GPU VRAM will generate far faster than a larger model that spills to system RAM. Our recommendations prioritize models that run at full speed on your hardware. Use the VRAM Calculator to verify exact memory requirements for any model and quantization format, and check what fits your exact GPU model by model.
Quick Guide: Model Size vs Quality
| Size | VRAM (Q4) | Size Class | Best For | Hardware |
|---|---|---|---|---|
| 1-3B | 1-3 GB | Tiny (edge) | Simple Q&A, autocomplete, edge devices | Any 4GB device |
| 7-9B | 5-6 GB | Small | Daily chat, coding assistance, RAG | 8GB GPU or Mac |
| 14B | 9-10 GB | Mid-size | Complex reasoning, debugging, analysis | RTX 3060 12GB |
| 32B | 20-22 GB | Large | Professional coding, writing, planning | RTX 4090 or 32GB Mac |
| 70B | 40-43 GB | Very large | Expert-level tasks, research, production | 64GB Mac or 2x GPU |
| 100B+ MoE | 50-60 GB | Large MoE | Largest open models, long context, vision | 128GB Mac or 3x GPU |
Size class is a parameter-count bucket, not a quality score: within one model family the larger checkpoint is usually the stronger one, but compare specific models on their model cards and public leaderboards. VRAM at Q4_K_M quantization with 4K context. Longer contexts add 0.5-4 GB. See hardware guide for detailed requirements.
Frequently Asked Questions
What is the best local AI model for coding in 2026?
For coding, Qwen 2.5 Coder 32B is the strongest code-specialised model that fits one 24GB card: Qwen reports 92.7% on HumanEval for the Instruct version and says its coding matches GPT-4o (Qwen2.5-Coder release, Nov 2024). It needs about 22 GB VRAM (RTX 4090 or 32GB Mac). For smaller setups, Qwen 2.5 Coder 7B (about 5.5 GB) is the code-specialised pick that fits an 8GB card. DeepSeek R1 14B (about 10 GB) adds chain-of-thought reasoning, which helps on multi-step debugging.
Which LLM runs best on 8GB VRAM?
With 8GB VRAM, your best options are Llama 3.1 8B (about 5.5 GB, versatile all-rounder), Mistral 7B (about 5 GB, reliable), Gemma 2 9B (about 6.5 GB, general tasks), and Qwen 2.5 Coder 7B (about 5.5 GB, code-specialised). All run at full GPU speed on an RTX 3070, RTX 4060, or 8GB Mac. Use Q4_K_M quantization for the best quality-to-size ratio.
What model should I use for RAG (retrieval-augmented generation)?
For RAG, Qwen 2.5 32B (about 22 GB VRAM) is the pick in the tool above for 24GB cards and 32GB Macs. If limited on VRAM, Llama 3.1 8B (5.5 GB) handles RAG well thanks to its 128K context window. For maximum context, Llama 4 Scout supports up to 10M tokens. Pair any of these with ChromaDB or Qdrant for the vector database.
How do I choose between model sizes (7B, 14B, 32B, 70B)?
Model quality scales with size but with diminishing returns: 7B models handle most everyday tasks adequately, 14B models add noticeable improvement in reasoning and nuance, and 32B and 70B models close much of the remaining gap to the closed frontier models. Always run the largest model your hardware can fit entirely in VRAM — a 14B model at full GPU speed beats a 70B model spilling to RAM.
Can I run AI models with just system RAM (no GPU)?
Yes, but much slower. Generation speed is capped by memory bandwidth divided by the model file size: on dual-channel DDR5-5600 (89.6 GB/s) a 7B model (4.4 GB file) has a ceiling of about 20 tokens/second, against about 230 tok/s on an RTX 4090 (1,008 GB/s). Real speed lands below the ceiling. Apple Silicon Macs sit in between thanks to unified memory: a base M4 (120 GB/s) has a ceiling of about 13 tok/s for Qwen 2.5 14B (9 GB file), and an Apple M3 Pro (150 GB/s) measured 28.5 tok/s on mistral:7b (Ollama 0.12.3, October 2026).
What is the best free AI model for creative writing?
Llama 3.3 70B is the largest dense open model most home setups can run, and the size helps with long-form prose — it handles fiction, marketing copy, poetry, and dialogue well. Its Ollama download is 43 GB, so it needs about 45 GB of VRAM or a 64GB Mac. For smaller setups, Gemma 2 9B (about 6.5 GB) and Qwen 2.5 32B (about 22 GB) are the creative-writing picks in the tool above. All are free, open-weight models you can run privately via Ollama.
Free to use — just keep the attribution link. Works on any site.
<iframe src="https://localaimaster.com/embed/model-recommender" width="100%" height="560" style="border:1px solid var(--line);border-radius:12px;max-width:680px" title="AI Model Recommender — Local AI Master" loading="lazy"></iframe>
<p style="font:13px/1.5 system-ui,sans-serif;max-width:680px;margin:6px 0 0"><a href="https://localaimaster.com/tools/model-recommender">AI Model Recommender</a> by <a href="https://localaimaster.com">Local AI Master</a></p>Know what to actually run on it
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Ready to Go Beyond Tutorials?
25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?
Go from reading about AI to building with AI
25 structured courses. Hands-on projects. Runs on your machine. Start free.
Written by
Barsha Mohapatra
Founder and Editor, Local AI Master · Founder, Little AI Master
Founder of Little AI Master and the editor behind every course and guide on Local AI Master. Years of tutoring students taught her where AI education actually fails.
Related Guides
Continue your local AI journey with these comprehensive guides
Grab the AI Starter Kit — career roadmap, cheat sheet, setup guide
No spam. Unsubscribe with one click.