Best AI Models for the RTX 4090
The 24GB tier that holds the 27–32B class at Q4 — footprints from arithmetic, speeds as bandwidth ceilings
The best AI model for the RTX 4090 is Qwen3.6 27B — ~17GB at Q4_K_M, which leaves working room inside 24GB, with a 68.9% SWE-bench Verified score that beats far larger models. The 4090 runs the full 32B class at Q4: Qwen 2.5 Coder 32B (~27 tok/s) for code, DeepSeek-R1 32B (~30 tok/s) for reasoning, and Flux.1 dev at full FP16 (~12s/image) for images. We do not own a 4090. VRAM figures here are arithmetic (~0.6GB per billion params at Q4_K_M) and throughput is a memory-bandwidth ceiling, not a measurement. The ceiling is real too: a 70B needs ~40GB and does not run usably on one card.
Models that run well on the RTX 4090
| Model | Size | Build | Memory used | Speed |
|---|---|---|---|---|
| WINNERQwen3.6 27B The headline pick — beats Qwen’s own 397B MoE on agentic coding (68.9% SWE-bench Verified), with ~7GB left for context. ollama pull qwen3.6:27b | 27B (dense) | Q4_K_M | ~17GB | ~30 tok/s (bandwidth estimate) |
| Qwen 2.5 Coder 32B The SOTA dense local coder (92.7% HumanEval) — the reason serious builds target 24GB. ollama pull qwen2.5-coder:32b | 32B | Q4_K_M | ~20GB | ~27 tok/s (bandwidth estimate) |
| DeepSeek-R1 32B Best local reasoning model — visible chain-of-thought at full 32B quality. ollama pull deepseek-r1:32b | 32B | Q4_K_M | ~20GB | ~30 tok/s (bandwidth estimate) |
| Qwen3-Coder 30B MoE agentic coder with a 256K native context — 30B-class reasoning at small-model speed. ollama pull qwen3-coder:30b | 30B (MoE, ~3B active) | Q4_K_M | ~18GB | ~50 tok/s (bandwidth estimate) |
| Command-R 35B Built for RAG and tool use — 128K context with citations. ollama pull command-r | 35B | Q4_K_M | ~21GB | ~25 tok/s (bandwidth estimate) |
| Flux.1 dev FP16 The reference image model with zero quantization — the flex only 24GB gets you. Close other GPU apps first. ComfyUI flux1-dev.safetensors (FP16) | 12B | FP16 | ~24GB | ~12s / image (bandwidth estimate) |
What won't fit on the RTX 4090
- ✗Llama 3.3 70B (needs ~40GB) — Needs 48GB — the classic route is 2×24GB (two used 3090s). On one card it only runs at Q2 with heavy offload; a 32B at Q4 answers better.
- ✗Qwen 2.5 72B (needs ~42GB) — Needs 48GB.
- ✗gpt-oss:120b (needs ~65GB+) — Workstation class — an 80GB card or a 128GB unified-memory box.
RTX 4090-specific notes
- →Where this page’s numbers come from: we do not own a 4090. Footprints are arithmetic — roughly 0.6GB per billion parameters at Q4_K_M — and every tok/s figure is a memory-bandwidth ceiling: 1,008 GB/s divided by the weights read per token. No engine reaches its ceiling, so read them as the speed a model cannot exceed on this card, not as a rate we recorded.
- →Power is the real installation question: 450W board power, NVIDIA’s minimum PSU recommendation is 850W, and millisecond transient spikes reach well past board power. An ATX 3.0 PSU with a native 12VHPWR/12V-2x6 cable — fully seated — is the no-drama setup; the early melted-connector cases traced overwhelmingly to partially seated cables.
- →The laptop trap: the “RTX 4090 Laptop” is a different chip entirely — AD103 (the desktop 4080’s die) with 16GB at 150W. Nothing in this page’s 17–24GB list fits it; a 4090 laptop is a 16GB-tier machine.
- →The 4090 runs the identical 24GB model list roughly 1.5–2× faster than an RTX 3090 — and not because of bandwidth, where the two cards are nearly matched (1,008 vs 936 GB/s). The gap comes from Ada’s much higher compute, clocks and far larger L2 cache, and it widens most on prompt processing. Either way, this is the fastest 24GB consumer card ever made.
- →Price sanity in the 2026 shortage: street prices run ~50%+ over MSRP across tiers and the RTX 5090 sits at $3,695+, so our GPU-prices guide’s advice applies — never pay panic prices. A used RTX 3090 runs this exact model list (slower) for far less, and if you’re about to spend 5090 money purely for capacity, a 128GB unified-memory box covers the 70B–200B class no consumer card can.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run qwen3.6:27bGo from "it runs" to actually building
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
What is the best LLM to run on an RTX 4090?
Qwen3.6 27B — ~17GB at Q4, roughly 30 tok/s by bandwidth estimate, and it out-scores far larger models on agentic coding. If your work is pure code, Qwen 2.5 Coder 32B; for hard reasoning, DeepSeek-R1 32B; for fast agent loops, the MoE qwen3-coder:30b at ~50 tok/s. All four figures are bandwidth ceilings for this card, not measured rates.
RTX 5070 Ti vs RTX 4090 for local AI?
The 4090’s 24GB runs the 27–32B class at full Q4 — Qwen3.6 27B, the 32B coder, R1 32B. The 5070 Ti’s 16GB caps out at gpt-oss:20b and the 14B class; newer silicon doesn’t change what fits. If your models live in the 27B+ range, the 4090 wins on capacity alone. Full breakdown in our RTX 5070 Ti vs 4090 comparison.
Can the RTX 4090 run a 70B model?
Not usably. A 70B needs ~40GB at Q4; on one 24GB card it only runs at Q2 with heavy system-RAM offload, at a few tok/s and visibly degraded quality. The proven route to 70B is two 24GB cards (2×3090 = 48GB). On a single 4090, a 32B at Q4 answers better than a squeezed 70B.
What PSU do I need for an RTX 4090 AI box?
NVIDIA’s minimum is 850W; an ATX 3.0 unit is worth it because the 4090’s millisecond transient spikes are what trip lesser supplies, not the average draw. Use the native 12VHPWR/12V-2x6 cable rather than adapter chains, and seat it fully — that’s the lesson of the melted-connector saga.
Related guides
Ready to Go Beyond Tutorials?
25 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?