AMD vs NVIDIA vs Intel GPUs: AI Runtime Support Matrix

Published April 23, 2026 · Updated August 23, 2026 · 12 min read

Short answer: the vendor question is really a runtime question. NVIDIA/CUDA is documented as supported by essentially every local-AI runtime. AMD/ROCm covers the mainstream stack — Ollama, llama.cpp, vLLM, PyTorch — but only on an explicit list of supported cards. Intel Arc has one well-trodden path (llama.cpp's SYCL backend plus Intel's own IPEX-LLM builds) and thinner coverage everywhere else. So: check that your runtime documents your vendor's backend first, then argue about VRAM and price.

The table below is the whole page in one screen. Everything after it explains a cell.

Which GPU vendor works with your AI runtime?

Verdicts describe what each project's own documentation supports today — this is a compatibility matrix, not a leaderboard.

How to read the verdicts:

• Works — a documented, first-class path. Follow the install guide and it runs.
• Flaky — a real path exists, but it is narrower: a short supported-card list, a manual environment variable, a separate vendor-maintained build, or an "experimental" label in the docs. Budget an afternoon.
• No — not a documented backend for that runtime. Change runtime, not GPU.

Two structural facts fall out of the table. First, llama.cpp is the vendor-agnostic escape hatch — it is the only row with four green cells, which is why "just use a GGUF" is the standard advice when someone's card is off the beaten path. Second, the CUDA-only tier is real but small: TensorRT-LLM and the research code that ships the week a paper drops. If neither of those is in your workflow, the NVIDIA lock-in argument is weaker than it sounds.

Primary sources worth reading before you buy: AMD's ROCm compatibility matrix (the authoritative list of which Radeon and Instinct parts are supported by which ROCm release), llama.cpp's SYCL backend documentation, Intel's IPEX-LLM repository, and NVIDIA's CUDA Toolkit documentation.

What is the AMD equivalent of nvidia-smi?

rocm-smi is the direct replacement, and newer ROCm releases ship amd-smi alongside it. On Intel, the equivalents are xpu-smi and intel_gpu_top. Here is the full translation table for the commands you actually use while debugging a local model:

The one that trips people up: rocminfo is the AMD command that answers "does ROCm see my card at all", and it prints a gfx architecture code (for example gfx1100) rather than a marketing name. That code is what you match against AMD's compatibility matrix, and what you feed to the override described in the ROCm unsupported-GPU workaround guide.

Which AMD graphics cards does ROCm support?

Consumer ROCm support is an allowlist, not a blanket "all Radeon cards". AMD publishes the list per ROCm release in the compatibility matrix linked above, and a card that is absent from it is not merely slower — it is unsupported, and the runtime may silently fall back to CPU.

The practical shape of it:

• RDNA 3 (RX 7900 XTX / XT / GRE) is the well-trodden consumer path — this is the generation most Ollama-on-Radeon guides were written against. The card-level detail is in the Radeon RX 7900 XTX for local AI guide.
• RDNA 4 (RX 9070 series) is newer, so check the matrix for the ROCm release you are installing rather than assuming. Card-specific notes: RX 9070 XT for local AI.
• Workstation and Instinct parts — the Radeon AI PRO R9700 and the MI300X, where 192GB of HBM3 changes which models fit on one card.
• Older GCN and RDNA 1 parts (Radeon VII, MI50, RX 5700) have mostly fallen off the list. They can still be coaxed into working — see the AMD MI50 local LLM guide — but Vulkan is often the saner route there than fighting ROCm.

If you are installing from scratch, the step-by-step is in the AMD ROCm local LLM setup guide.

Which Intel Arc cards work for local AI?

Two cards account for nearly every Intel question: the Arc B580 (12GB, Battlemage) and the Arc A770 (16GB, Alchemist). Both have a documented path to running local models, and it is the same path: llama.cpp's SYCL backend, or Intel's IPEX-LLM, which ships prebuilt portable packages so you do not have to compile oneAPI yourself.

What you are trading away is breadth, not basic capability. Mainstream GGUF text generation is fine. The friction shows up at the edges — image-generation custom nodes, fine-tuning libraries, and anything that assumes a CUDA device string. Card-level detail: Intel Arc B580 for local AI and Intel Arc A770 for local AI.

AMD or Intel for AI when NVIDIA is out of budget?

Decide on VRAM first, ecosystem second. A 24GB Radeon holds model classes a 12GB Arc cannot hold at any speed, and no amount of software polish fixes "the weights do not fit". That makes the choice mostly mechanical:

• You want to run 27B–32B class models, or 70B across two cards → AMD, because 20GB and 24GB Radeon cards exist and Intel's consumer parts top out lower.
• You want the cheapest working entry point for 7B–14B models → Intel Arc is genuinely competitive on price per gigabyte, with the software caveats above.
• Your workflow is ComfyUI plus a pile of custom nodes → NVIDIA, or accept real weekends of debugging. This is the workload where the "Flaky" cells hurt most.
• Your workflow is Ollama or llama.cpp and nothing else → both AMD and Intel are legitimate. This is the case the table above is green for.

How much VRAM do you actually need?

This part is arithmetic, not opinion. Weights in gigabytes ≈ parameters (billions) × bytes per parameter, where 4-bit quantisation is roughly 0.5 bytes per parameter, 8-bit is 1, and FP16 is 2.

Those are floors, computed from the formula above. Real GGUF files land somewhat higher because some tensors are kept at higher precision, and you still need headroom for the KV cache, which grows with context length. Do not plan to fill the card exactly. The per-model detail lives in the VRAM requirements guide and the Ollama model RAM and VRAM table.

What each card holds and how fast its memory is

For token generation on a quantised model, memory bandwidth is the usual bottleneck — the GPU has to stream every active weight for each token it emits. Bandwidth is not a mystery number: it is bus width (bits) × data rate (Gbps) ÷ 8, so the last column below is derived from the two before it.

Memory configuration and board power are the vendors' published specifications; board partner cards vary, so check the product page for the exact model you are buying. Street prices move week to week and are deliberately not in this table — check current listings, and read the used GPU buying guide before paying for a second-hand 3090 or 4090.

Two things worth noticing. The RX 7900 XTX and the RTX 4090 sit within about 5% of each other on memory bandwidth (960 vs 1,008 GB/s) with the same 24GB capacity — which is why the AMD-versus-NVIDIA argument at that tier is a software argument, not a hardware one. And the RTX 5090's 32GB is the only single-card consumer configuration that holds a 70B model at Q4 (≈35 GB by the arithmetic above) without splitting it across two GPUs; the two-card alternatives are costed out in the dual 3090 vs 5090 comparison.

Why there is no tokens-per-second chart on this page. We do not own all ten of these cards, and a benchmark table we did not run is worse than no table at all. Bandwidth is the honest proxy, and your own numbers are two commands away.

How do I benchmark my own card?

Both of these work identically on CUDA, HIP, SYCL and Vulkan builds, so you can compare vendors on your own hardware rather than trusting anyone's chart:

# llama.cpp: prompt-processing and token-generation rates, averaged over repeats
llama-bench -m ./models/llama-3.1-8b-q4_k_m.gguf -p 512 -n 128

# Ollama: prints eval count and eval rate after the response
ollama run llama3.1:8b --verbose

Run each a few times after a warm-up pass — the first run after a cold model load includes disk read time and will look artificially slow.

Which vendor should you pick?

Pick NVIDIA if you want the widest runtime support with zero research, you fine-tune as well as infer, you need TensorRT-LLM or day-one support for new research code, or your time is worth more than the price gap.

Pick AMD if your exact card appears in the ROCm compatibility matrix, you are on Linux, capacity per dollar matters more than ecosystem breadth, and your workload is mainstream text generation.

Pick Intel Arc if you want the cheapest working entry point, 7B–14B models cover your needs, and you are comfortable using llama.cpp's SYCL build or Intel's IPEX-LLM packages rather than whatever the top search result assumes.

Do not pick AMD or Intel if you need TensorRT-LLM, you run ComfyUI on Windows with exotic custom nodes, or you routinely clone research repos the week they publish — those ship CUDA-first, and often CUDA-only.

What breaks first on each stack?

ROCm reports the GPU as unsupported. Confirm the gfx target with rocminfo and check it against AMD's matrix. If the card is genuinely absent from the list, the HSA_OVERRIDE_GFX_VERSION environment variable is the standard workaround — the full override guide explains which substitutions are safe and which produce wrong answers rather than errors.

An AMD card runs far slower than expected. Usually it silently fell back to CPU. Check that the runtime log names your GPU, not "CPU", before assuming the hardware is the problem.

CUDA and ROCm installed on the same machine fight each other. They do not coexist cleanly. Pick one toolchain per machine.

A GGUF refuses to load on the Intel SYCL backend. Try a plain Q4_0 or Q4_K_M build of the same model before blaming the card — backend coverage of the more exotic K-quants and i-quants is uneven across non-CUDA backends.

Mixing vendors in one box. Fine for running two separate models in two separate processes; not fine for splitting one model across an NVIDIA and an AMD card. A single tensor-parallel job cannot span two vendor runtimes.

Buying used. Stress the card with a real sustained workload for half an hour and watch for driver resets in dmesg or Windows Event Viewer. Consumer GeForce cards do not report ECC errors, so there is no shortcut query that proves memory health.

Where the specific answers live

This page is the map. The territory:

• AMD ROCm local LLM setup — the actual install, start to finish.
• ROCm unsupported GPU: the HSA override — what to do when your Radeon is not on the list.
• Radeon RX 7900 XTX for local AI — the 24GB AMD card, in detail.
• RX 9070 XT for local AI — the RDNA 4 option.
• Intel Arc B580 and Intel Arc A770 — the two Arc cards worth considering.
• RTX 3090 for local AI and RTX 4090 vs 3090 — the used-NVIDIA question.
• Cheapest 70B build: dual 3090 vs 5090 — when one card is not enough.
• AI hardware requirements, AI server build under $1500 and the workstation cooling guide — the rest of the machine.

Common questions

Is ROCm production-ready for inference?

For mainstream inference on cards AMD actually lists as supported, yes — Ollama, llama.cpp and vLLM all document working ROCm paths. The caveat is the word "listed". ROCm's reliability question is less "is the software good" and more "is my specific GPU in the matrix", which is why that link appears three times on this page.

Can I mix an NVIDIA and an AMD card in one machine?

Yes, physically and for separate workloads. No, for one inference job. Tensor parallelism requires a single vendor runtime across all participating GPUs.

What about Apple Silicon?

Different category — unified memory means the CPU and GPU share one pool, so capacity questions work differently. See the Apple M4 for AI guide, the Mac local AI setup guide and MLX vs CUDA.

Do NPUs replace a discrete GPU?

Not for workhorse inference. Laptop NPUs are designed for always-on, low-power small models rather than sustained token generation. The NPU comparison ranks Intel, Qualcomm, AMD and Apple on-chip accelerators against each other.

What about data-center cards?

H100, A100, L40S and MI300X all outclass consumer cards and cost accordingly. For personal use the price-to-benefit ratio does not work; the MI300X deep dive covers the one thing they genuinely unlock, which is enormous single-card memory capacity.

---

The short version

NVIDIA's advantage in 2026 is not raw silicon, it is that every row of the compatibility matrix says "Works". AMD's ROCm covers the mainstream runtimes properly, on a published list of cards you must check before buying. Intel Arc has one solid path through llama.cpp and IPEX-LLM and is the cheapest way in. The order of operations that saves the most regret: confirm your runtime supports the backend, confirm the model fits in VRAM, then compare bandwidth — and measure the rest yourself.

---

Want more hardware comparisons? Join the LocalAIMaster newsletter for build guides and GPU coverage.