RX 7900 XTX vs RTX 3090: Which 24GB Card for Local AI

Buy the RX 7900 XTX if your stack is Ollama, llama.cpp, vLLM or ComfyUI — all four document gfx1100 support by name, and AMD's ROCm 7.14 compatibility matrix lists the card outright, so you do not need HSA_OVERRIDE_GFX_VERSION at all. Buy a used RTX 3090 if you need ExLlamaV2/EXL2, NVLink memory pooling, or any other CUDA-only tool, because none of those has an AMD equivalent. On the hardware itself there is almost nothing in it: 960 GB/s of peak memory bandwidth versus 936 GB/s, 24 GB versus 24 GB, 355 W versus 350 W. This is a software decision wearing a hardware costume.

The reason the question keeps getting asked is that both cards sit at the same place in the market — the cheapest route to 24 GB of VRAM — and the folk wisdom about them is three years stale. "ROCm doesn't work" was a reasonable summary in 2023. It is not a reasonable summary of a stack whose current compatibility matrix names the RX 7900 XTX as a supported product, and whose QLoRA library ships prebuilt gfx1100 wheels for both Linux and Windows.

What follows is built entirely from vendor spec sheets and the runtimes' own documentation, with every claim linked. There are no tok/s figures on this page, and the section on speed explains exactly why that is a deliberate choice rather than an omission.

What Do You Actually Get for the Money?

Every number in this table comes from AMD's or NVIDIA's own published specification for the card. Where a figure is derived rather than published, the arithmetic is shown in the row.

Four things in that table are worth stopping on.

The bus width is derived, not published. AMD's product page gives "up to 20 Gbps" and "up to 960 GB/s" but no interface width. Divide: 960 GB/s ÷ (20 Gbit/s ÷ 8 bits per byte) = 960 ÷ 2.5 = 384 bits. Both cards are 384-bit, which is why they land within 2.6% of each other on bandwidth (960 ÷ 936 = 1.026).

AMD's "effective memory bandwidth" figure is not comparable. The same product page also quotes "up to 3,500 GB/s effective memory bandwidth". That number includes hits in the 96 MB Infinity Cache. It is a real thing and it helps real workloads, but it is not the DRAM figure and it must not be lined up against NVIDIA's 936 GB/s. The comparable pair is 960 versus 936.

The tensor-throughput comparison is genuinely ambiguous. NVIDIA's GA102 whitepaper splits FP16 tensor throughput by accumulate precision — 142 TFLOPS accumulating in FP16, 71 TFLOPS accumulating in FP32. AMD publishes a single "Peak Half Precision (FP16 Matrix) Performance: 123 TFLOPs" and does not state the accumulate precision. So the honest reading is: the 3090 is ahead on FP16-accumulate matrix maths and clearly ahead on INT8/INT4 (284 and 568 TOPS versus 123 and 246), while the 7900 XTX is ahead on plain FP32 vector maths (61.4 versus 35.6 TFLOPS). Anyone who tells you one card is flatly "2× faster at AI" is picking a row.

NVIDIA's tensor numbers above are the dense ones. Table 9 of the whitepaper quotes a second, doubled figure for each, achievable only with the Ampere structured-sparsity feature: 284 TFLOPS FP16, 568 TOPS INT8, 1,136 TOPS INT4. Sparsity does not apply to a stock GGUF quant you downloaded off Hugging Face, so the dense column is the one that describes your evening.

Which Card Is Faster for Local LLMs?

We are not going to publish a tok/s table, and the reason is that we do not have both cards. There is no public benchmark that runs both under identical conditions — same quant, same context length, same batch size, same runtime build — and stitching two different people's numbers together produces a comparison that looks authoritative and means nothing. What we can do is show you the arithmetic that governs the answer, so you can sanity-check any figure you find elsewhere.

Single-user token generation is memory-bandwidth bound. To emit one token, a dense model reads essentially its entire weight set out of VRAM once. So the ceiling is bandwidth divided by the bytes you have to read:

• A ~20 GB quantised model on the 7900 XTX: 960 GB/s ÷ 20 GB = 48 tokens/s ceiling
• The same model on the 3090: 936 GB/s ÷ 20 GB = 46.8 tokens/s ceiling

That is a 1.2 tok/s theoretical gap, and no real runtime achieves the ceiling — KV-cache traffic, kernel efficiency and memory-controller overheads all take a cut. Whatever the real numbers are, they will be close, because the physics is close. If you see a benchmark claiming one of these cards generates tokens twice as fast as the other on the same dense model, the two runs were not comparable.

Prompt processing is a different story. Prefill is compute-bound rather than bandwidth-bound: it multiplies the whole prompt through the model in parallel, which is exactly what tensor/matrix units are for. That is where the 3090's 284 INT8 TOPS against the 7900 XTX's 123 TOPS should show up. If you paste 8,000-token documents into a local model all day, the NVIDIA card has a structural advantage in the part of the job you wait for. If you type short chat messages, you will almost never touch that path.

Both cards hold the same models. Using the standard rule of thumb that a Q4_K_M GGUF costs roughly 0.6 GB per billion parameters: a 32B model is about 32 × 0.6 = 19 GB, which fits in 24 GB with room left for a KV cache. A 70B is about 70 × 0.6 = 42 GB, which does not fit on either card and needs two GPUs or CPU offload — see our breakdown of the cheapest way to run a 70B locally for what that actually costs.

Both cards also land in the same place on the wall socket: 355 W typical board power versus 350 W TGP is a rounding error, though AMD asks for a 800 W PSU with more than 65 A on the 12 V rail while NVIDIA asks for 750 W. Our local AI power consumption measurements cover what idle and sustained inference actually cost per month.

Is ROCm Survivable on a 7900 XTX in 2026?

For this specific card, yes — and the strongest evidence is that AMD lists it by name. The ROCm 7.14.0 compatibility matrix, published 2026-07-16, includes "AMD Radeon RX 7900 XTX (gfx1100)" among its supported devices, alongside gfx1101, gfx1102, the RDNA 4 targets gfx1200/gfx1201, the Ryzen APU targets, and the CDNA line.

This matters more than it sounds like it should, because most of the horror stories you will find about AMD and local AI are about cards that are not on that list. An RX 6600 is gfx1032 and has never been officially supported; getting it working means setting HSA_OVERRIDE_GFX_VERSION to lie to the runtime about which chip you have, and living with whatever that substitution does to your output. The 7900 XTX has none of that problem. If you are on a 7900 XTX and you have an override set, it is more likely to be causing a bug than fixing one.

The practical consequences of being on the supported list:

• Prebuilt binaries exist for your architecture. You are not compiling code objects for a target that ships in no wheel.
• The distro matrix is narrow but real. ROCm 7.14.0's certified Linux versions include Ubuntu 26.04 (GA kernel 7.0) and Ubuntu 24.04.4 (GA kernel 6.8), plus the RHEL line down to 8.10. Install one of those point releases rather than an interim build — this is the single most common source of self-inflicted ROCm pain.
• The driver has to match the userspace. Ollama's documentation is blunt about this: it "bundles ROCm 7 linux libraries which require a compatible ROCm 7 kernel driver," and the fix when they mismatch is to "upgrade to the ROCm v7 driver using the amdgpu-install utility."

For the full install walkthrough — repo setup, group membership, verification with rocminfo — our AMD ROCm local LLM setup guide is the companion page, and the RX 7900 XTX deep dive covers tuning that specific card once it is running.

Which Runtimes Support Which Card?

This is the table the decision actually turns on. Each row is what the project's own documentation says, not an inference from what someone reported working.

Read that table as a filter rather than a scoreboard. Seven of the eight rows are a tie or close to it. The row that decides your purchase is whichever one you cannot live without.

Two rows deserve a footnote. The FlashAttention row is the most commonly overstated fact in this whole comparison: AMD's fork does cover RDNA, but through the Triton backend — the Composable Kernel backend, which is the one people mean when they say "FlashAttention on AMD," names only MI200 and MI300 in the README. And the ExLlamaV2 row is not a gap you can engineer around. EXL2 is the fastest single-GPU INT4 path there is, and if that is your workflow, our ExLlamaV2 and TabbyAPI guide is a CUDA-only document; a 7900 XTX simply does not run it.

What Actually Goes Wrong on the AMD Side?

The failures you will hit are almost never "the GPU is not supported." They are permissions, driver-version skew and multi-GPU quirks — and they present identically to unsupported hardware, which is why the internet is full of people setting overrides that cannot help them. These are the strings Ollama's own troubleshooting documentation records:

failure during GPU discovery ... error="failed to finish discovery before timeout"
bootstrap discovery took duration=30s

That pair is the driver-skew signature, not a support problem. The documentation explains it directly: Ollama bundles ROCm 7 Linux libraries that require a compatible ROCm 7 kernel driver, and the resolution is upgrading the driver with amdgpu-install.

The second most common cause is device access. Per the same doc, "AMD GPU access typically requires video and/or render group membership to access the /dev/kfd device," and inside a container you have to "pass additional --group-add ... arguments to the container so it can access the required devices" — with numeric GIDs, because the host's group names do not exist inside the image.

# Is the runtime even seeing the GPU?
rocminfo | grep -i gfx          # expect gfx1100 on a 7900 XTX
groups                          # expect render and video

# Turn the logs up before guessing
AMD_LOG_LEVEL=3 OLLAMA_DEBUG=1 ollama serve

Both of those variables are documented: AMD_LOG_LEVEL=3 raises the AMD HIP/ROCm libraries to info level, and OLLAMA_DEBUG=1 adds detail during GPU discovery.

There is one genuinely AMD-shaped hazard worth knowing about before you buy a second card: the documentation has a dedicated note about gibberish responses "when models load across multiple AMD GPUs on Linux," pointing at AMD's own multi-GPU known-issues page. A single 7900 XTX is not affected. A pair of them is a configuration you should test with a fixed prompt and a fixed seed before trusting.

And NVIDIA is not failure-free — it just fails differently. The same troubleshooting document lists CUDA initialisation errors "3 (not initialized), 46 (device unavailable), 100 (no device), 999 (unknown)", suggests CUDA_ERROR_LEVEL=50 for more diagnostics, and offers sudo nvidia-modprobe -u or unloading and reloading nvidia_uvm as fixes. If your mental model is "CUDA just works and ROCm is a science project," the vendor's own bug list disagrees. Our AMD vs NVIDIA vs Intel runtime support matrix covers where each vendor's stack currently stands.

Can You Fine-Tune on Either Card?

Yes, on both — and this is the single most out-of-date belief about AMD in local AI. The bitsandbytes documentation, which is the library behind essentially every QLoRA tutorial you have read, now states plainly: "All features are supported for both consumer RDNA devices and Data Center CDNA products."

That is backed by shipped wheels rather than an aspiration. The published build matrix lists, for Linux x86-64 on ROCm 7.14.0, RDNA targets including gfx1100 — the 7900 XTX — and for Windows x86-64 on ROCm 7.14.0, gfx1100 again. On the NVIDIA side the CUDA wheels include sm86, and the feature table sets LLM.int8() at compute capability 7.5 or higher (the 3090 is 8.6) with NF4/FP4 quantisation available from 6.0 upward.

Where the AMD side is still visibly thinner is attention kernels during training. The ROCm FlashAttention fork routes RDNA through its Triton backend, and the Composable Kernel backend — the more mature of the two — documents MI200 and MI300 only. That does not stop you fine-tuning; it does mean the fastest attention path is not the one you get.

The realistic summary: both cards will train a LoRA on a 7B-to-14B model at 24 GB. Neither will train anything large. If fine-tuning is the primary reason you are buying, the ecosystem depth argument favours NVIDIA, and it favours it on tooling and tutorials more than on the core library.

Does the 3090's NVLink Still Matter?

It matters if and only if you intend to buy two of them. NVIDIA's 30 Series page documents an NVLink bridge for SLI-ready 30 Series products and lists the 3090 as supporting it; AMD's RX 7900 XTX specification sheet publishes no equivalent card-to-card link, so a dual-7900-XTX box communicates over PCIe.

For a single card this is worth exactly nothing, and most buyers reading this page are buying one card. For the dual-GPU 48 GB build — the cheapest honest route to a 70B at Q4, per the 42 GB arithmetic above — it is a real architectural advantage on the NVIDIA side, and it is the main reason the used 3090 keeps its reputation among people running larger models. It is also why the 3090 is more interesting than the 4090 for this particular job, a comparison our RTX 3090 local AI guide goes into.

Note the physical cost of that plan. NVIDIA's Founders Edition 3090 is a 3-slot card, 313 mm long and 138 mm wide; AMD's reference 7900 XTX is 2.5 slots and 287 mm. Two 3-slot cards need a motherboard and case that were designed for it, and partner-card dimensions vary from the reference in both directions.

What Changes if You Are on Windows?

Less than it used to, but the AMD path is still the one with caveats. Three sourced data points:

• ComfyUI ships experimental Windows builds for AMD, and its README notes those builds "have less hardware support than the builds above but they work on windows," limited to RDNA 3, 3.5 and 4 architectures. The 7900 XTX is RDNA 3, so it is inside that window — but "experimental" is the project's word, not ours.
• bitsandbytes publishes Windows x86-64 ROCm wheels for gfx1100 at both ROCm 7.2.1 and 7.14.0, so QLoRA on Windows AMD is a supported configuration on paper.
• AMD's "ROCm on Radeon" documentation — the consumer-focused track — currently covers releases through 7.2.1 and scopes itself to "Radeon GPUs (9000 & select 7000 Series)". The consumer stack trails the main ROCm line.

On the NVIDIA side, Windows CUDA is simply the mainstream configuration for every one of these tools, which is not a technical argument so much as an ecosystem one — and ecosystem arguments are precisely what you are paying the CUDA premium for.

What Do These Cards Cost Right Now?

We are not going to quote you a price, because we do not have verified sold-listing data and a stale number here would be worse than none. Used GPU pricing moves weekly, it is regional, and the memory shortage that has been distorting the whole market since late 2025 makes any figure written in August 2026 unsafe to trust in October. The site has published invented hardware figures in the past and has removed them; this is not the page to restart that habit.

What you can do in two minutes is get a number that is actually current:

1. On eBay, search the exact card, then filter to Sold Items and Completed Items. Asking prices are fiction; sold prices are data.
2. Exclude the outliers on both ends — the "for parts" listings and the sealed-collector listings — and take the median of what is left.
3. Do it for both cards on the same day, in your own country. A cross-border comparison is not a comparison.
4. Add the delta in PSU requirements if you are near the line: AMD asks for 800 W, NVIDIA for 750 W.

Two structural points that do not move week to week and are worth folding into whatever numbers you find. First, the 7900 XTX can still be bought new, with a warranty, while the 3090 is an Ampere-generation card you are buying second-hand from a stranger — factor in the risk of a card that spent three years mining. Our used GPU buying guide for AI covers the inspection checklist for exactly that. Second, both cards are priced inside a market distorted by memory costs; the context in our analysis of why GPU prices are so high explains why the ordering between new and used has been unusually unstable.

So Which One Should You Buy?

Buy the RX 7900 XTX if:

• Your stack is Ollama, llama.cpp, vLLM or ComfyUI. All four name gfx1100 in their docs.
• You want a card with a warranty rather than a six-year-old part with unknown history.
• You are on Linux, on one of ROCm's certified distro releases, and comfortable reading a compatibility matrix before installing.
• You are buying one card, not two.
• You want the higher FP32 vector throughput (61.4 versus 35.6 TFLOPS) for non-LLM work.

Buy a used RTX 3090 if:

• You need ExLlamaV2/EXL2. This is the clearest single reason on the page, and it has no workaround.
• You plan to run two cards for a 70B, where NVLink is a genuine architectural advantage.
• Long-prompt prefill is your bottleneck — the 3090's 284 INT8 TOPS versus 123 is the compute-bound side of the job.
• You want the broadest possible tutorial and custom-node ecosystem, particularly for image generation, and you would rather not be the person debugging the AMD path.
• You are fine-tuning frequently enough that "the mature attention kernels" is a phrase that means something to you.

It genuinely does not matter which you pick if you run one card, chat with 8B-to-32B models through Ollama or llama.cpp, and never touch EXL2. Both hold the same models, both read memory at roughly the same rate, both draw roughly the same power. In that scenario, buy whichever is cheaper on the day, and spend the saved hour on picking better models to run instead of on the card.

Honest Limitations

• Neither card was tested for this page. Every figure is from a vendor specification sheet or a project's documentation, and every one is linked below. We do not own a 7900 XTX or a 3090 and are not going to pretend otherwise.
• There are no tok/s numbers here on purpose. No public benchmark runs both cards under matched conditions, and combining two unmatched runs produces a confident-looking number that is not true. The bandwidth arithmetic above is what we can honestly stand behind.
• The FP16 matrix comparison is not clean. AMD does not publish the accumulate precision behind its 123 TFLOPS figure, and NVIDIA's equivalent is either 142 or 71 depending on which accumulate you pick. Treat that row as directional.
• Documentation moves. The ROCm compatibility matrix cited here is 7.14.0, dated 2026-07-16; the vLLM, Ollama, llama.cpp, ComfyUI and bitsandbytes pages are living documents on main branches. Check the current version of any row that decides your purchase.
• Partner cards differ from reference. The 287 mm / 2.5-slot and 313 mm / 3-slot dimensions are AMD's reference and NVIDIA's Founders Edition. Board partners ship longer, thicker and differently-cooled versions of both.

Sources

• AMD Radeon RX 7900 XTX product specifications — compute units, stream processors, AI accelerators, clocks, FP32/FP16/INT8/INT4 peaks, 24 GB GDDR6, 20 Gbps, 960 GB/s, 96 MB Infinity Cache, 355 W, 800 W PSU, 287 mm, 2.5 slots, launch date
• AMD GPU specifications reference (ROCm docs) — the RX 7900 XTX to gfx1100 mapping
• ROCm 7.14.0 compatibility matrix (published 2026-07-16) — RX 7900 XTX listed as supported, plus the certified Linux releases
• NVIDIA GeForce RTX 3090 product page — 10,496 CUDA cores, 3rd-gen Tensor Cores, 24 GB GDDR6X, 384-bit, 350 W, 750 W system power, power connectors, NVLink, Founders Edition dimensions
• NVIDIA Ampere GA102 GPU architecture whitepaper (v2) — Table 9: 936 GB/s at 19.5 Gbps, 328 Tensor Cores, 35.6 FP32 TFLOPS, 142/71 FP16 tensor TFLOPS, 284 INT8 TOPS, 568 INT4 TOPS, 6,144 KB L2, 350 W TGP, 28.3 billion transistors
• vLLM GPU installation documentation — "Radeon RX 7900 series (gfx1100/1101)" on ROCm 6.3+, and the CUDA compute capability 7.5 minimum
• Ollama GPU support documentation and Ollama troubleshooting documentation — the AMD and NVIDIA support tables, the ROCm 7 driver requirement, the discovery-timeout strings, /dev/kfd group membership, container --group-add, the multi-AMD-GPU gibberish note, and the CUDA error codes
• llama.cpp build documentation — the HIP build with -DGPU_TARGETS=gfx1100 and the CUDA build with -DCMAKE_CUDA_ARCHITECTURES="86;89"
• ComfyUI README — the rocm7.2 and cu130 PyTorch wheel indexes, and the experimental Windows AMD builds limited to RDNA 3/3.5/4
• bitsandbytes installation documentation — full RDNA support, the ROCm 7.14.0 Linux and Windows wheel targets including gfx1100, and the CUDA compute-capability feature table
• ROCm/flash-attention — the Composable Kernel backend scoped to MI200/MI300 and the Triton backend covering CDNA and RDNA
• ExLlamaV2 — the CUDA Toolkit requirement and the absence of any ROCm or AMD path