RTX 5090 vs RTX 4090 for AI: 32GB vs 24GB, Which to Buy

Where do these numbers come from?

Every figure below is either a published NVIDIA specification (linked) or an arithmetic result you can reproduce from those specs using the formula shown beside it. Nothing here is a measured benchmark: we do not own either card.

That is the more useful answer anyway. Real throughput depends on your quantisation, context length, runtime version, driver and how much of the model actually lives on the GPU, so a single borrowed tokens-per-second number tells you almost nothing about your own machine. The bandwidth and VRAM arithmetic tells you the shape of the answer, and it does not go stale when llama.cpp ships a new release.

RTX 5090 vs RTX 4090: the published specifications

Source: NVIDIA's official product pages for the GeForce RTX 5090 and GeForce RTX 4090.

How much faster is the RTX 5090 than the 4090 for local LLMs?

Token generation on a single GPU is memory-bandwidth bound, not compute bound. To emit one token the runtime has to read the model's active weights out of VRAM once. That gives a hard arithmetic ceiling:

And you need the model's footprint. A useful rule of thumb for the most common quantisation, Q4_K_M in llama.cpp/GGUF:

Put the two together and the comparison falls out:

The one line worth taking away: on every model that fits both cards, the ratio between those two columns is identical — 1.78. It has to be, because it is nothing more than 1,792 ÷ 1,008. The 5090's ceiling is 78% above the 4090's, full stop. If you see a claimed real-world speedup materially above 78%, it is not coming from the memory system, and you should ask what else changed between the two runs (quantisation, context length, batch size, or how many layers were on the GPU at all).

Mixture-of-experts models are the interesting exception. Mixtral 8x7B has 46.7B total parameters but activates roughly 12.9B per token, so its speed tracks the active slice while its VRAM requirement tracks the total. That is why it is fast on a 5090 and simply absent on a 4090.

Which models actually fit in 24GB vs 32GB?

How much context does the extra 8GB actually buy?

The KV cache is the other consumer of VRAM, and it grows linearly with context length:

Worked example, Llama 3.1 8B (32 layers, 8 grouped-query KV heads, 128-dimension heads, fp16 cache):

2 × 32 × 8 × 128 × 2 bytes = 128 KB per token. So a 32K context costs about 4GB, and a 128K context costs about 16GB — on top of the 4.8GB of weights. That is the real reason the 8GB matters: it is roughly 64K extra tokens of context on an 8B model, or the difference between a 32B model with a scratch-sized window and a 32B model you can hand a whole file to.

Quantising the KV cache to 8-bit halves those figures, which is often a better trade than dropping the weights to a lower quant.

What does each card cost to run?

Board power is a published figure; running cost is arithmetic:

At 4 hours a day and $0.15/kWh, using each card's full rated board power as a worst case:

Treat those as ceilings. Total graphics power is the board's rated limit, and single-stream LLM generation is memory-bound rather than compute-saturating, so neither card holds its rated draw continuously during chat-style inference. Image generation and batch serving get much closer to it.

The PSU line matters more than the electricity line for most buyers: NVIDIA recommends a 1,000W system supply for the 5090 against 850W for the 4090. If your build is on an 850W unit, budget for a replacement before you compare sticker prices.

Price per unit of the thing you are actually buying

Both "better" columns for the 5090 come from dividing MSRP by a vendor spec — do the same division with whatever you can actually pay today, because street prices for both cards have spent long stretches well above MSRP and the ranking flips easily. If a 5090 is 40% over list and a used 4090 is under list, the 4090 wins both rows.

What about the RTX 5080 and a used RTX 3090?

Two things jump out of that table, and both are the kind of thing raw bandwidth arithmetic is good at exposing:

1. The RTX 5080's problem is not speed, it is 16GB. Its bandwidth ceiling on an 8B model is within a few percent of a 4090's. What it cannot do is hold a 27B or 32B model with a usable context window. If you were choosing on token rate alone it would look like a bargain; you are really choosing on VRAM.
2. A used RTX 3090 sits at 93% of a 4090's memory bandwidth with identical VRAM. For memory-bound LLM generation that is much closer than gaming benchmarks suggest, which is why the 3090 has stayed the value pick for VRAM per dollar. It loses on power efficiency, on prompt processing (which is compute-bound), and on warranty. Our used GPU buying guide covers sourcing one without getting burned.

Who should buy what?

Buy the RTX 5090 if:

• You want a 27B–32B model resident with a long context window
• You run Mixtral 8x7B or similar MoE models that need ~28GB
• Prompt processing throughput matters to you (long documents, RAG, code review) — that workload is compute-bound, and the 5090's 33% CUDA core advantage shows up there rather than in the bandwidth ratio
• You already have, or are budgeting for, a 1,000W PSU

Buy the RTX 4090 if:

• Your daily driver is a 7B–14B model
• You can find one at or below MSRP, or used
• Your existing 850W PSU is staying put
• You are content trimming context on a 32B model

Buy a used RTX 3090 if:

• Budget is the binding constraint and you want 24GB anyway
• You accept 350W of older-generation power draw and no warranty
• Generation speed matters more to you than prompt-processing speed

Buy neither if you need 70B on one card — that requires ~48GB, which means two cards, a workstation card, or a far more aggressive quantisation. If you are considering pairing GPUs, note that an external card over Thunderbolt or OCuLink is also an option; the eGPU bandwidth analysis works through what an external link does and does not cost you.

Upgrade paths

The pattern is consistent: below 24GB, upgrade for VRAM. At 24GB, upgrade only if a specific model you want is stuck behind the 32GB line.

Frequently asked questions

How much faster is the RTX 5090 than the 4090 for AI inference?

For token generation, the arithmetic ceiling is 78% higher, because that is the memory-bandwidth ratio (1,792 GB/s vs 1,008 GB/s). Real-world gains land below that ceiling and vary with quantisation, context length and runtime. For prompt processing — which is compute-bound rather than bandwidth-bound — the relevant ratio is the 33% CUDA core increase instead.

Is the RTX 5090 worth $400 more than the RTX 4090?

It is worth it if a model you actually run needs more than 24GB — a 27B or 32B model with real context, or Mixtral 8x7B. If your daily driver is 7B–14B, the extra 8GB buys you context headroom you may never use, and the 4090 is the better value. Check what you run against the fit table above before deciding.

What models can the RTX 5090 run that the 4090 cannot?

Anything between roughly 24GB and 30GB of weights plus cache. In practice that means Mixtral 8x7B at Q4 (28GB), a 32B dense model with a long context rather than a trimmed one, and higher-quality quants of 27B models. It does not include 70B at Q4 — that is 42GB and fits on neither.

Can the RTX 5090 run Llama 70B?

Not entirely on the GPU. 70B at Q4_K_M is about 42GB against 32GB of VRAM. You can run it with layers offloaded to system RAM, but the offloaded portion streams at DDR5 speeds rather than GDDR7 speeds, and the result is dramatically slower than the on-card ceiling. Two 24GB cards or one 48GB card is the clean answer.

What power supply do I need for the RTX 5090?

NVIDIA's published recommendation is a 1,000W system power supply, against 850W for the 4090. The card draws through a 16-pin 12V-2x6 connector; seat it fully and use the cable that shipped with the PSU rather than a daisy-chained adapter.

Does the RTX 5090 support NVLink?

No. NVLink was dropped from GeForce after the RTX 3090. Two 5090s communicate over PCIe, which is fine for tensor-parallel inference but is not the high-bandwidth bridge that older multi-GPU guides assume.

How does GDDR7 improve AI performance?

It raises memory bandwidth from 1,008 GB/s to 1,792 GB/s, and since token generation is bandwidth-bound, that is the single largest lever on generation speed. The wider 512-bit bus contributes as much as the memory generation itself. It does nothing for VRAM capacity, which is the other half of the decision.

Should I wait for an RTX 5090 or buy a 4090 now?

Decide on VRAM, not on release date. If nothing you run exceeds 24GB, a 4090 at or below MSRP is the better purchase today and will stay capable. If you are repeatedly hitting the 24GB wall, waiting for a 5090 at a sane price is worth it — but do not pay a large premium over list for a card whose generation-speed advantage is a known 78% ceiling.

Does the RTX 5090 support GGUF, AWQ and EXL2?

Yes. All the mainstream quantisation formats — GGUF via llama.cpp and Ollama, AWQ and GPTQ via vLLM, EXL2 via ExLlamaV2 — run on Blackwell. Note that support for a brand-new architecture usually lands in the runtime a release or two after launch, so keep CUDA and your runtime current.

Next steps

1. Compare every GPU for local AI across the full price range
2. Work out how much VRAM your models actually need before choosing between 24GB and 32GB
3. Understand what each quantisation format costs you in quality and footprint
4. Run DeepSeek R1 locally once the card is in
5. Set up Ollama to serve it

---

Specifications on this page come from NVIDIA's published product pages. Every derived figure states its formula so you can check the arithmetic against your own numbers.