Best AI Models for the RTX 4090
The 24GB tier’s benchmarks were run on this exact card — the 27–32B class with nothing scaled or estimated
The best AI model for the RTX 4090 is Qwen3.6 27B — ~17GB at Q4_K_M and ~30 tokens/sec measured on this card, with a 68.9% SWE-bench Verified score that beats far larger models. The 4090 runs the full 32B class at Q4: Qwen 2.5 Coder 32B (~27 tok/s) for code, DeepSeek-R1 32B (~30 tok/s) for reasoning, and Flux.1 dev at full FP16 (~12s/image) for images. Every number on this page was benchmarked on an RTX 4090 — no scaling, no estimates. The ceiling is real too: a 70B needs ~40GB and does not run usably on one card.
Models that run well on the RTX 4090
| Model | Size | Build | Memory used | Speed |
|---|---|---|---|---|
| WINNERQwen3.6 27B The headline pick — beats Qwen’s own 397B MoE on agentic coding (68.9% SWE-bench Verified), with ~7GB left for context. ollama pull qwen3.6:27b | 27B (dense) | Q4_K_M | ~17GB | ~30 tok/s (measured on the 4090) |
| Qwen 2.5 Coder 32B The SOTA dense local coder (92.7% HumanEval) — the reason serious builds target 24GB. ollama pull qwen2.5-coder:32b | 32B | Q4_K_M | ~20GB | ~27 tok/s (measured on the 4090) |
| DeepSeek-R1 32B Best local reasoning model — visible chain-of-thought at full 32B quality. ollama pull deepseek-r1:32b | 32B | Q4_K_M | ~20GB | ~30 tok/s (measured on the 4090) |
| Qwen3-Coder 30B MoE agentic coder with a 256K native context — 30B-class reasoning at small-model speed. ollama pull qwen3-coder:30b | 30B (MoE, ~3B active) | Q4_K_M | ~18GB | ~50 tok/s (measured on the 4090) |
| Command-R 35B Built for RAG and tool use — 128K context with citations. ollama pull command-r | 35B | Q4_K_M | ~21GB | ~25 tok/s (measured on the 4090) |
| Flux.1 dev FP16 The reference image model with zero quantization — the flex only 24GB gets you. Close other GPU apps first. ComfyUI flux1-dev.safetensors (FP16) | 12B | FP16 | ~24GB | ~12s / image (measured on the 4090) |
What won't fit on the RTX 4090
- ✗Llama 3.3 70B (needs ~40GB) — Needs 48GB — the classic route is 2×24GB (two used 3090s). On one card it only runs at Q2 with heavy offload; a 32B at Q4 answers better.
- ✗Qwen 2.5 72B (needs ~42GB) — Needs 48GB.
- ✗gpt-oss:120b (needs ~65GB+) — Workstation class — an 80GB card or a 128GB unified-memory box.
RTX 4090-specific notes
- →This page’s numbers are native: our 24GB-tier benchmarks (the LLM tok/s figures and the ~12s Flux render) were all run on an RTX 4090. Most GPU pages have to scale from other cards — this one doesn’t.
- →Power is the real installation question: 450W board power, NVIDIA’s minimum PSU recommendation is 850W, and millisecond transient spikes reach well past board power. An ATX 3.0 PSU with a native 12VHPWR/12V-2x6 cable — fully seated — is the no-drama setup; the early melted-connector cases traced overwhelmingly to partially seated cables.
- →The laptop trap: the “RTX 4090 Laptop” is a different chip entirely — AD103 (the desktop 4080’s die) with 16GB at 150W. Nothing in this page’s 17–24GB list fits it; a 4090 laptop is a 16GB-tier machine.
- →The 4090 runs the identical 24GB model list roughly 1.5–2× faster than an RTX 3090 — and not because of bandwidth, where the two cards are nearly matched (1,008 vs 936 GB/s). The gap comes from Ada’s much higher compute, clocks and far larger L2 cache, and it widens most on prompt processing. Either way, this is the fastest 24GB consumer card ever made.
- →Price sanity in the 2026 shortage: street prices run ~50%+ over MSRP across tiers and the RTX 5090 sits at $3,695+, so our GPU-prices guide’s advice applies — never pay panic prices. A used RTX 3090 runs this exact model list (slower) for far less, and if you’re about to spend 5090 money purely for capacity, a 128GB unified-memory box covers the 70B–200B class no consumer card can.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run qwen3.6:27bGo from "it runs" to actually building
All 519 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
What is the best LLM to run on an RTX 4090?
Qwen3.6 27B — ~17GB at Q4, ~30 tok/s measured on the 4090, and it out-scores far larger models on agentic coding. If your work is pure code, Qwen 2.5 Coder 32B; for hard reasoning, DeepSeek-R1 32B; for fast agent loops, the MoE qwen3-coder:30b at ~50 tok/s. All four were benchmarked on this card.
RTX 5070 Ti vs RTX 4090 for local AI?
The 4090’s 24GB runs the 27–32B class at full Q4 — Qwen3.6 27B, the 32B coder, R1 32B. The 5070 Ti’s 16GB caps out at gpt-oss:20b and the 14B class; newer silicon doesn’t change what fits. If your models live in the 27B+ range, the 4090 wins on capacity alone. Full breakdown in our RTX 5070 Ti vs 4090 comparison.
Can the RTX 4090 run a 70B model?
Not usably. A 70B needs ~40GB at Q4; on one 24GB card it only runs at Q2 with heavy system-RAM offload, at a few tok/s and visibly degraded quality. The proven route to 70B is two 24GB cards (2×3090 = 48GB). On a single 4090, a 32B at Q4 answers better than a squeezed 70B.
What PSU do I need for an RTX 4090 AI box?
NVIDIA’s minimum is 850W; an ATX 3.0 unit is worth it because the 4090’s millisecond transient spikes are what trip lesser supplies, not the average draw. Use the native 12VHPWR/12V-2x6 cable rather than adapter chains, and seat it fully — that’s the lesson of the melted-connector saga.
Related guides
Ready to Go Beyond Tutorials?
20 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?