Best AI Models for the RTX 5090: What 32GB Unlocks
What the extra 8GB over a 24GB card actually buys — quant levels, resident text encoders and context, with every footprint computed
The best AI model for the RTX 5090 is Qwen3.6 27B, but the 32GB changes how you run a model more than which models you can run. A 24GB card holds Qwen3.6 27B at Q4_K_M (~17GB); 32GB holds the same weights at Q6_K (~22GB) or Q8_0 (~29GB) — 27 billion params × ~0.82 and ~1.06 GB per billion. The other three unlocks are a dense 32B at Q5_K_M (~23GB) instead of Q4, Flux.1 dev at FP16 with its T5 text encoder resident (~24GB + ~4.7GB), and context — a 32K window on a 32B costs exactly 8GB of KV cache, which is the 5090’s entire memory advantage over a 4090. One thing this page will not give you: a tokens-per-second number. We have not benchmarked a 5090, so every speed below is a bandwidth roofline with the arithmetic shown, never a measurement we did not take.
Models that run well on the RTX 5090
| Model | Size | Build | Memory used | Speed |
|---|---|---|---|---|
| WINNERQwen3.6 27B The headline unlock. Same model a 4090 runs, two or three quant levels higher: 27 × ~0.82 = ~22GB at Q6_K, 27 × ~1.06 = ~29GB at Q8_0. Neither loads on 24GB once you add a KV cache. Pick the q6_K or q8_0 tag on ollama.com/library/qwen3.6 | 27B (dense) | Q6_K or Q8_0 | ~22GB (Q6_K) / ~29GB (Q8_0) | roofline 1,792 ÷ 22 ≈ 81 tok/s at Q6_K, ÷ 29 ≈ 62 at Q8_0 — a ceiling, not a measurement |
| Qwen 2.5 Coder 32B A 4090 runs this coder at Q4_K_M (~19GB) with roughly 4GB left over. The 5090 runs it a quant level up and still has ~8GB for context — the difference is repo-scale prompts, not a different model. Pick the q5_K_M tag on ollama.com/library/qwen2.5-coder | 32B | Q5_K_M | ~23GB (32 × ~0.71) | roofline 1,792 ÷ 23 ≈ 78 tok/s — ceiling only |
| DeepSeek-R1 32B Fits a 24GB card too, so the weights are not the unlock — the KV cache is. This model is architecturally Qwen 2.5 32B (64 layers, 8 KV heads, 128 head dim), so FP16 KV costs 256KB per token: ~12GB free on a 5090 buys ~48K tokens of reasoning chain versus ~16K on a 4090. ollama pull deepseek-r1:32b | 32B | Q4_K_M | ~19GB (32 × ~0.60) | roofline 1,792 ÷ 19 ≈ 94 tok/s — ceiling only |
| Qwen3-Coder 30B 32GB-only at Q6_K: 24GB holds it at Q4 (~18GB) and no higher. An MoE is the best argument for capacity over bandwidth — you are paying for the VRAM to hold all 30B, while only a fraction is read each token. Pick the q6_K tag on ollama.com/library/qwen3-coder | 30B (MoE, ~3B active) | Q6_K | ~25GB (30 × ~0.82) | MoE — only ~3B params are read per token, so the bandwidth roofline lands in the hundreds and stops being the limit. Real speed is set by kernel and attention overhead, and we have no figure for that on a 5090. |
| Llama 3.3 70B (Q2_K) The capability flex, with a caveat we are not going to bury: Q2_K is an aggressive quantization and costs real output quality. Q3_K_S computes to ~30GB (70 × ~0.43) and leaves nothing for KV cache. Benchmark it against a 32B at Q4 on your own prompts before adopting it. ollama pull llama3.3:70b-instruct-q2_K | 70B | Q2_K | ~26GB (70 × ~0.37) | roofline 1,792 ÷ 26 ≈ 69 tok/s — ceiling only |
| Flux.1 dev FP16 + T5-XXL The clearest practical unlock. On 24GB the transformer alone fills the card, so ComfyUI evicts and reloads the T5 encoder every time the prompt changes. 32GB keeps both resident. ComfyUI: flux1-dev.safetensors (FP16) + t5xxl_fp8_e4m3fn.safetensors | ~12B transformer + 4.7B text encoder | FP16 transformer, FP8 encoder | ~29GB (12 × 2 = ~24GB, plus 4.7 × 1 = ~4.7GB) | diffusion is compute-bound rather than bandwidth-bound, so the roofline method used elsewhere on this page does not apply — and we have no 5090 render timing to quote |
| gpt-oss:20b Included as the honest counterexample: this runs comfortably on a 16GB card and beautifully on 24GB. If gpt-oss:20b and the 14B class are your daily drivers, the 5090 buys you speed you may not need, not capability. ollama pull gpt-oss:20b | 20.9B (MoE, ~3.6B active) | MXFP4 | ~11GB of weights (20.9 × ~0.53); our 32GB VRAM guide lists ~14–16GB resident once context is allocated | MoE again — bandwidth is not the binding constraint, and we have no measurement for this card |
What won't fit on the RTX 5090
- ✗Llama 3.3 70B (Q4_K_M) (needs ~42GB) — 70 × ~0.60 = ~42GB of weights before any context. Full-quality 70B is still a 48GB job — two 24GB cards, or two 5090s. The 32GB card gets you a 70B at Q2, not a 70B.
- ✗Qwen3-Coder-Next 80B (needs ~48GB+) — 80 × ~0.60 = ~48GB at Q4 with zero KV cache, so 64GB in practice. The big MoE coders are dual-GPU or unified-memory territory.
- ✗gpt-oss:120b (needs ~65GB+) — Roughly 117B params even in its native MXFP4 format. This is an 80GB workstation card or a 128GB unified-memory box.
RTX 5090-specific notes
- →Read this before you trust any 5090 tok/s number, here or elsewhere: we do not own this card and have published no first-party 5090 benchmark. Every speed figure on this page is a roofline — memory bandwidth divided by the weights read per token — which is a hard ceiling that no inference engine actually reaches, because attention, sampling and kernel launches all cost time the roofline ignores. The bandwidth itself is arithmetic from NVIDIA’s spec: a 512-bit bus running 28 Gbps GDDR7 is 512 × 28 ÷ 8 = 1,792 GB/s, against the 4090’s 384 × 21 ÷ 8 = 1,008 GB/s — 1.78× more. If another page quotes you an exact 5090 tok/s without naming the rig, the engine, the quant and the context length, treat it as invented.
- →How every footprint here was computed, so you can audit it: parameter count × bytes per weight, using llama.cpp’s published bits-per-weight for each k-quant — Q2_K ≈ 2.96 bits (~0.37GB per billion params), Q3_K_S ≈ 3.41 (~0.43), Q4_K_M ≈ 4.83 (~0.60), Q5_K_M ≈ 5.67 (~0.71), Q6_K ≈ 6.56 (~0.82), Q8_0 = 8.5 (~1.06). That is weights only. The KV cache is a separate bill and it is the one people forget.
- →The 8GB is a context budget, not a parameter budget — and the arithmetic is unusually tidy. Qwen 2.5 32B (and the DeepSeek-R1 32B distilled from it) has 64 layers, 8 KV heads and a head dimension of 128, so one token of FP16 KV cache costs 2 (for K and V) × 64 × 8 × 128 × 2 bytes = 262,144 bytes, or 256KB. A 32,768-token context therefore costs 32,768 × 256KB = exactly 8GB. The 5090’s entire memory advantage over a 4090 is one 32K context on one 32B model. That is a real upgrade if you feed models whole files; it is nothing at all if you chat in 4K windows.
- →The laptop trap is the worst in the lineup: NVIDIA’s own laptop spec lists the GeForce RTX 5090 Laptop GPU at 24GB GDDR7 on a 256-bit bus with a 95–150W graphics power range. That is 8GB less memory, half the bus width and roughly a quarter of the desktop card’s 575W, all under the same name. Nothing on this page that needs more than 24GB applies to a 5090 laptop — our RTX 4090 page is much closer to its real model list.
- →The Blackwell software trap costs more people a weekend than any hardware limit. Every GeForce RTX 50-series card reports compute capability 12.0, which is sm_120, and PyTorch’s CUDA 12.6 wheels are compiled for compute capabilities 5.0 through 9.0 — the list stops at Hopper. Any project that pins an older torch in requirements.txt will throw “RuntimeError: CUDA error: no kernel image is available for execution on this device” on a perfectly working 5090. Ollama ships its own CUDA runtime and is largely unaffected; ComfyUI, vLLM and anything with compiled extensions (xformers, flash-attn, bitsandbytes, sageattention) are where it bites. The fix is a CUDA 13.x PyTorch build, and the second trap is that a library can lack sm_120 even when PyTorch has it.
- →Power, from NVIDIA’s spec page rather than from folklore: 575W total graphics power, a recommended 1000W system supply, and either 4× PCIe 8-pin through the included adapter or a single 600W PCIe Gen 5 cable. The 40-series connector lesson carries over unchanged — use a native cable rather than an adapter chain where you can, and seat it fully. Also worth planning for: LLM decoding is a sustained load measured in minutes, not the bursty load a gaming PSU and case airflow were sized around.
- →Price sanity before you buy the 8GB: our GPU-prices guide tracked the 5090 Founders Edition at $3,695 on Newegg in mid-July 2026 against a $1,999 MSRP, with Amazon listings near $4,329. At those numbers the honest comparison is not 5090 versus 4090 — it is “$3,700 for eight more gigabytes” versus a used 24GB card that runs six of the seven rows above, or a 128GB unified-memory box at around $3,099 that holds the 70B–120B models neither card can touch at any quant.
Quick start
curl -fsSL https://ollama.com/install.sh | shollama run qwen3.6:27bollama psGo from "it runs" to actually building
All 561 chapters — running local models, RAG, agents, fine-tuning — plus the Python Lab and every course added later.
Frequently asked questions
What is the best AI model for an RTX 5090?
Qwen3.6 27B, and on this card the interesting decision is the quantization rather than the model. At Q4_K_M it is ~17GB and runs on any 24GB card; the 5090 holds it at Q6_K (~22GB) or Q8_0 (~29GB), which is the whole point of the extra memory. For code, Qwen 2.5 Coder 32B at Q5_K_M (~23GB) is the quant step a 4090 cannot take. For long reasoning chains, DeepSeek-R1 32B at Q4 plus a genuinely large KV cache.
How many tokens per second does an RTX 5090 get?
We have not benchmarked one, so we are not going to tell you. What we can give you is the ceiling, which you can compute for any model yourself: token generation on a dense model reads the whole weight set once per token, so the upper bound is memory bandwidth divided by footprint. The 5090 has 1,792 GB/s (512-bit × 28 Gbps ÷ 8), so a 20GB model cannot exceed 1,792 ÷ 20 ≈ 90 tok/s no matter what. Real engines land materially below that, because attention, sampling and kernel overhead are not free. Any page that quotes an exact 5090 figure without naming the rig, engine, quant and context length is guessing.
Can an RTX 5090 run a 70B model?
At Q2_K, yes — 70 × ~0.37 = ~26GB of weights, which fits with a modest context, and it is the first single consumer card that can hold a 70B entirely in VRAM. But be clear about what you are buying: Q2_K is aggressive quantization with real quality cost, and the next step up, Q3_K_S, computes to ~30GB and leaves no room for a KV cache. Full-quality Q4_K_M is ~42GB and needs 48GB. On one 5090, a dense 32B at Q4 usually answers better than a 70B squeezed to Q2 — test both on your own prompts rather than taking either side of that on faith.
Is the RTX 5090 worth it over an RTX 4090 for local AI?
It depends entirely on whether you are memory-bound today. The 5090 has 1.78× the memory bandwidth on paper and 8GB more VRAM, and that 8GB is the real product: higher quants on 27–32B models, Flux FP16 with its text encoder resident, and roughly a 32K rather than an 8K context on a 32B. If your daily models are 14B or smaller, or gpt-oss:20b, a 24GB card already runs everything you use and the upgrade buys speed alone. At the street prices in our GPU-prices guide, that is an expensive way to buy speed.
Is the RTX 5090 Laptop the same as the desktop RTX 5090?
No, and the gap is larger than any other name collision in the 50-series. NVIDIA lists the RTX 5090 Laptop GPU at 24GB GDDR7 on a 256-bit bus with a 95–150W graphics power range, against the desktop card’s 32GB on 512-bit at 575W. The laptop chip is a 24GB machine, so it loses the Q5/Q6 32B rows, the Q8 27B row and resident-encoder Flux. Its power limit is also chosen by the laptop maker, so two machines with a “5090” inside can perform very differently.
Why does my RTX 5090 throw “no kernel image is available for execution on this device”?
Because something in your Python stack was compiled without sm_120. Every RTX 50-series card reports compute capability 12.0, and PyTorch’s CUDA 12.6 wheels are built for compute capabilities 5.0 through 9.0 — they contain no Blackwell kernels, so the error is guaranteed rather than bad luck. Install a CUDA 13.x PyTorch build. If the error persists after that, a second compiled wheel is the culprit — xformers, flash-attn, bitsandbytes, sageattention or a ComfyUI custom node shipping its own kernels. Ollama bundles its own CUDA runtime and rarely hits this.
Related guides
Ready to Go Beyond Tutorials?
20 structured courses with hands-on chapters - build RAG chatbots, AI agents, and ML pipelines on your own hardware.
Was this helpful?