Ollama Not Using GPU: 8 Causes, One Test for Each

Short answer: run ollama ps while the model is loaded and read the PROCESSOR column. "100% GPU" means it works. "100% CPU" means Ollama never found a usable GPU — check the server log for no compatible GPUs were discovered, then go to causes 2-8. Anything in between, like 48%/52% CPU/GPU, means the GPU was found and the model did not fit — that is cause 1, and it has nothing to do with your drivers. Those are two completely different problems and almost every guide on the internet mashes them together into "check your drivers."

Written against Ollama v0.32.14 (released 2026-08-15), the current release at the time of writing, with every log line and environment variable below traced to Ollama's own docs or to a linked issue in its tracker. We call the version out because GPU discovery and its log output have been reworked more than once across releases — if you are on something older, the exact log lines may differ.

---

Start Here: The One Command That Splits the Problem

ollama ps is the diagnostic. Not nvidia-smi.

Load a model, leave it loaded, and run this in a second terminal:

# terminal 1 — keep this open so the model stays resident
ollama run qwen3:8b "write a haiku about VRAM"

# terminal 2 — while the model is still loaded
ollama ps

You get something like:

NAME        ID              SIZE      PROCESSOR          UNTIL
qwen3:8b    <model-id>      6.5 GB    48%/52% CPU/GPU    4 minutes from now

Ollama's own FAQ documents the three states of that PROCESSOR column: 100% GPU (entirely on the GPU), 100% CPU (entirely in system memory), and a mixed percentage meaning the model "was loaded partially onto both the GPU and into system memory."

Why not nvidia-smi? Because it answers a different question. It tells you a process exists and how much VRAM is allocated — it does not tell you how many layers are on the card, and utilisation sampled between token batches can read near zero on a perfectly healthy run. People stare at 0% utilisation and conclude the GPU is unused when the real problem is that only half the layers are there.

Note the SIZE column too. That number is weights plus KV cache plus compute buffers, not the download size of the model. It is routinely 1-4 GB larger than the file you pulled, and that gap is the single most common reason a "4 GB model" does not fit on a 12 GB card.

Now branch:

---

"No Compatible GPUs Were Discovered" — What That Log Line Actually Means

no compatible GPUs were discovered means discovery ran at server start and came back with an empty list. Ollama never attempted to use a card, so nothing you change at run time will help — and it is logged at INFO, not ERROR, which is why people grep their logs and find nothing.

The line looks like this, and the exact wording is stable enough that it is worth searching for verbatim:

time=<timestamp> level=INFO source=gpu.go:<line> msg="looking for compatible GPUs"
time=<timestamp> level=INFO source=gpu.go:<line> msg="no compatible GPUs were discovered"

A GitHub search of the ollama/ollama tracker on 2026-08-22 returns 152 issues and pull requests containing that exact string, against 592 for the looking for compatible GPUs line that precedes it. The pair is the single most-reported symptom of this whole problem, and the reason it gets misdiagnosed is the log level: grep -i error and grep -i fail both miss it completely.

# the grep that actually catches it, on all three platforms
journalctl -u ollama --no-pager | grep -E "looking for compatible GPUs|no compatible GPUs|amdgpu is not supported|compute capability"

# macOS
grep -E "compatible GPUs|metal|amdgpu" ~/.ollama/logs/server.log

# Docker
docker logs <container-name> 2>&1 | grep -E "compatible GPUs|amdgpu is not supported"

The line immediately above it is the one that names your cause. Discovery logs why each backend was rejected before it prints the empty result. These are the three you are most likely to see, quoted as they appear in the tracker:

Note the second line prints two things you need: gpu_type= is your card's target, and the bracketed list is what Ollama actually built for. On Windows the same warning comes from amd_windows.go instead of amd_linux.go. Older Ollama builds format it as msg="amdgpu is not supported" gpu=0 gpu_type=gfx1035 library=/opt/rocm/lib supported_types="[...]" — same information, different punctuation.

---

Cause 1: Partial Offload — The KV Cache Ate Your VRAM

If PROCESSOR shows a split, this is your cause, and the fix is context length — not drivers. At Ollama's default 4096-token context a Qwen3-8B-class model needs about 0.56 GB of f16 KV cache. At 32768 tokens the same model needs about 4.5 GB. That is the difference between fitting on a 12 GB card and spilling layers to the CPU.

The KV cache is allocated per loaded model and scales linearly with context. The formula is not a mystery:

KV bytes per token = 2 (K and V) x layers x kv_heads x head_dim x bytes_per_element

Plug in published model configs and you get concrete numbers. Layer counts and KV-head counts below are from each model's own config.json on Hugging Face; the arithmetic is ours, at f16 (2 bytes):

Two things fall out of that table. First, an 8× context increase is an 8× KV increase — there is no clever compression happening by default. Second, the 32B model burns 8 GB of KV cache alone at 32K context, before a single weight is loaded. That is why people with 24 GB cards find a 32B Q4 model (~20 GB of weights) suddenly half on the CPU the moment they raise the context.

The one-command test: reload with the default context and see if the split goes away.

# stop the server, then start it with an explicit small context
OLLAMA_CONTEXT_LENGTH=4096 ollama serve

If ollama ps now reads 100% GPU, you have confirmed it: context was the problem, not hardware.

The three fixes, in the order we would try them:

1. Lower the context to what you actually need. Most chat use never touches 8K. Ollama's documented default is 4096 tokens, settable per-server with OLLAMA_CONTEXT_LENGTH, per-session with /set parameter num_ctx 4096, or per-request with "num_ctx" in the API.
2. Quantise the KV cache. Ollama documents OLLAMA_KV_CACHE_TYPE with three values: f16 (default), q8_0 (about half the memory of f16) and q4_0 (about a quarter). Setting q8_0 turns that 8 GB Qwen3-32B cache into roughly 4 GB. This is a quality trade, and Ollama's own documentation says so — it notes that a quantised KV cache reduces memory at some cost to precision, and that the impact varies by model. Test both values on your own prompts before adopting either.
3. Drop a quantisation level on the weights. Q4_K_M instead of Q6_K buys back more room than any tuning flag.

# half the KV cache, one line
OLLAMA_KV_CACHE_TYPE=q8_0 OLLAMA_CONTEXT_LENGTH=8192 ollama serve

Before you buy anything, run the numbers for the exact model and context you want on our VRAM calculator, and cross-check against the Ollama model RAM/VRAM table.

One more thing that catches people: if two models are resident at once, both are holding VRAM. Set OLLAMA_MAX_LOADED_MODELS=1, or OLLAMA_KEEP_ALIVE=0 to unload immediately after each request. Ollama keeps models in memory for five minutes by default.

---

Cause 2: Your Driver Is Below the Floor

Ollama's hardware docs state driver 550+ as the general requirement, and driver 570+ for GPUs with compute capability 5.0-6.2. Below the floor, Ollama falls back to CPU silently while nvidia-smi keeps working perfectly — which is precisely why this one wastes so many evenings.

The one-command test:

nvidia-smi --query-gpu=name,driver_version,compute_cap --format=csv

One line, three answers: which card, which driver, which compute capability. Compare the driver number to the floor. If your card is Maxwell or Pascal (compute capability 5.0-6.2 — GTX 900 and GTX 10-series), the number you need is 570, not 550.

The trap is long-term-support distro drivers. A stable Ubuntu box that has been happily running an older branch for two years will show a healthy nvidia-smi and give you 100% CPU in Ollama. Upgrading the driver is the whole fix.

# Debian/Ubuntu — check what your distro actually offers before installing
apt-cache search '^nvidia-driver-[0-9]' | sort

After any driver change, reboot. Not "restart Ollama" — reboot. And on Linux, if you updated the kernel recently, the NVIDIA modules may need rebuilding before they load at all; our Ollama troubleshooting guide covers the DKMS side of that.

---

Cause 3: The Card Is Below the Compute-Capability Cutoff

Ollama supports NVIDIA GPUs with compute capability 5.0 or higher. Below 5.0 there is no fix and no flag — the card will never be used, and everything else in this guide is a waste of your time. Get the number from the compute_cap field in the command above.

Here is the supported table as published in Ollama's hardware docs, so you can find your card:

Source: Ollama's docs/gpu.mdx, read on 2026-08-18 against the v0.32.x tree.

Worth being precise here, because the forums are not: Pascal is not dropped. GTX 10-series cards at compute capability 6.1 are still in the supported table. What changed is that they sit in the 570+ driver band, so an old driver strands them on CPU in a way that reads exactly like "unsupported." If you are running a 10-series card, check the driver before you believe anyone telling you the card is dead. And check the docs for your Ollama version — this matrix has moved before and can move again.

If your card is genuinely below 5.0, the honest answer is that CPU inference on a small model is your path: a 3B at Q4 is usable on a modern CPU, a 70B is not. The Ollama system requirements page has the RAM-to-model-size mapping for CPU-only boxes.

---

Cause 4: GPU Discovery Failed at Server Start

If the driver and the card are both fine, the next thing to read is the server log from the moment the server started — not from when you ran the model. GPU discovery happens once, at startup, and the failure is recorded there with a numeric CUDA error code.

The one-command test — read the log:

# Linux (systemd)
journalctl -u ollama --no-pager | grep -i -E "gpu|cuda|rocm|vulkan|library" | tail -40

# macOS
grep -i -E "gpu|metal|library" ~/.ollama/logs/server.log | tail -40

# Docker
docker logs <container-name> 2>&1 | grep -i -E "gpu|cuda|library" | tail -40

On Windows the log locations are documented as %LOCALAPPDATA%\Ollama for server logs, %LOCALAPPDATA%\Programs\Ollama for binaries and %HOMEPATH%\.ollama for models. To get verbose output there, quit the app from the tray menu first, then:

$env:OLLAMA_DEBUG="1"
& "ollama app.exe"

The error codes, straight from Ollama's troubleshooting docs:

The documented first move for codes 3 and 999 on Linux is the UVM module:

sudo nvidia-modprobe -u
# or, to force a clean reload
sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm

For more detail than the log gives you, Ollama documents CUDA_ERROR_LEVEL=50 for detailed CUDA logging, and sudo dmesg | grep -i nvidia for anything the kernel is complaining about (Xid errors, falling-off-the-bus messages). On AMD, the equivalents are AMD_LOG_LEVEL=3 for HIP/ROCm detail and OLLAMA_DEBUG=1 during discovery; the docs note that "failure during GPU discovery" or bootstrap timeouts on AMD usually mean an outdated driver, with ROCm v7 as the current Linux requirement. If discovery instead prints amdgpu is not supported with a gpu_type= field, the driver is fine and you are in cause 8 — a gfx target Ollama did not build for.

# maximum-detail restart, one line
OLLAMA_DEBUG=1 CUDA_ERROR_LEVEL=50 ollama serve 2>&1 | tee /tmp/ollama-gpu.log

---

Cause 5: The Container or Service Cannot See the Device

This is the top cause when nvidia-smi works in your shell but Ollama reports no GPU — the process Ollama runs as is not the process you tested with.

The one-command test for Docker: run nvidia-smi inside the container.

docker exec -it <container-name> nvidia-smi

If that fails inside but works on the host, Ollama is not the problem. You need the NVIDIA Container Toolkit on the host and the GPU flag on the run command:

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Ollama's docs also call out a cgroup driver mismatch as a distinct Docker cause — the fix is adding "exec-opts": ["native.cgroupdriver=cgroupfs"] to /etc/docker/daemon.json and restarting Docker. If you run Ollama behind a web UI in Compose, the same flags have to be on the Ollama service specifically; see our Ollama + Open WebUI Docker setup.

The one-command test for systemd: check what the service user can see, not what you can see.

sudo -u ollama nvidia-smi

On AMD this is the usual failure. Ollama's docs are explicit that GPU access requires video and/or render group membership for /dev/kfd:

sudo usermod -aG video,render ollama
sudo systemctl restart ollama

Two smaller variants of the same class of problem, both documented: a noexec /tmp on hardened Linux boxes stops Ollama extracting its runtime libraries — set OLLAMA_TMPDIR to a writable, executable location. And if you run more than one card, a stale device filter can hide one of them; the multi-GPU specifics are in our Ollama multi-GPU setup guide.

---

Cause 6: Ollama Picked Vulkan Instead of CUDA

On a machine with both backends installed, check the startup log for which one was selected before changing anything. Ollama's hardware docs describe Vulkan as additional support on Windows and Linux, with device selection via GGML_VK_VISIBLE_DEVICES and an off switch at OLLAMA_VULKAN=0.

This matters because Vulkan and CUDA are not equivalent paths. A run can look "GPU accelerated" in ollama ps while behaving nothing like the CUDA numbers you read in a benchmark article — the backend that got selected is a variable most benchmark posts never state.

The one-command test:

journalctl -u ollama --no-pager | grep -i -E "vulkan|cuda|rocm" | head -20

If the log shows Vulkan on a machine with a working CUDA stack, force CUDA and compare:

OLLAMA_VULKAN=0 ollama serve

Then re-run the same prompt and compare tokens per second. If the numbers are identical, backend selection was not your problem — put the flag back and move on. We are deliberately not publishing a "Vulkan is X% slower" figure here. We do not own this hardware, and any single number would depend on the card, the driver and the model — your own before-and-after on the same prompt is the only measurement that means anything.

---

Cause 7: Something Is Explicitly Forcing CPU

Check for leftover environment variables before you rebuild anything — a stale CUDA_VISIBLE_DEVICES=-1 in a shell profile or a systemd drop-in produces a perfect 100% CPU with no error at all. Ollama's docs document -1 as the value that forces CPU-only mode for CUDA, with ROCR_VISIBLE_DEVICES doing the same job on AMD.

The one-command test:

# what the running server actually inherited (Linux)
sudo tr '\0' '\n' < /proc/$(pgrep -f 'ollama serve' | head -1)/environ | grep -i -E "cuda|rocr|ollama|vulkan|hsa"

That reads the environment of the live process, which is the only version that counts. Your interactive shell can be spotless while the systemd unit carries a two-year-old override. The usual suspects:

• CUDA_VISIBLE_DEVICES=-1 or ROCR_VISIBLE_DEVICES=-1 — forces CPU
• OLLAMA_LLM_LIBRARY=cpu_avx2 — pins the CPU backend and bypasses autodetection entirely. Ollama's troubleshooting docs list cpu_avx2 (best performance), cpu_avx and cpu (most compatible). It is a genuinely useful debugging tool and a genuinely common thing to leave behind after debugging
• HSA_OVERRIDE_GFX_VERSION set to a wrong target on AMD — documented as the override for mapping unsupported GPUs onto a similar LLVM target, and it will happily map you onto something that does not work. Setting it on a card that is already supported is a common self-inflicted wound; see cause 8 for which cards need one and which do not

# find the systemd override that is doing it
systemctl cat ollama | grep -i environment

The WSL2 variant. If Ollama is installed inside WSL2, it needs the WSL CUDA driver path to work at all — test it with wsl nvidia-smi from PowerShell. And WSL2 has a second, sneakier failure: it can allow allocations to spill into shared system memory rather than failing, which reads as "GPU is being used" while running at CPU-like speed. If your WSL2 numbers are far below what the same card does natively, install Ollama on Windows directly and compare; our Ollama Windows installation guide covers the native path.

---

Cause 8: On AMD, Your gfx Target Is Not One Ollama Built For

If the log says amdgpu is not supported and names a gpu_type=, your driver is fine and your card is fine — Ollama simply did not ship a compiled code object for your chip's architecture. HSA_OVERRIDE_GFX_VERSION makes the runtime report a different architecture, which works when your chip is a configuration variant of a supported one and fails badly when it is not.

This is by far the biggest gap in most "Ollama not using GPU" guides, and the tracker shows why: a GitHub search of ollama/ollama on 2026-08-22 returns 966 issues and pull requests mentioning HSA_OVERRIDE_GFX_VERSION and 1,227 mentioning ROCm. The most-discussed hardware threads in the repository are AMD ones — #738 "AMD GPU & ROCm support" at 323 comments, #2453 "Add support for older AMD GPU gfx803, gfx802, gfx805 (e.g. Radeon RX 580, FirePro W7100)" at 222, PR #6282 "AMD integrated graphic on linux kernel 6.9.9+, GTT memory, loading freeze fix" at 195, and #2637 "Integrated AMD GPU support" at 171.

Step 1 — get your real gfx target. Do not guess it from the marketing name.

rocminfo | grep -i gfx
# or, if rocminfo is not installed
journalctl -u ollama --no-pager | grep -o 'gpu_type=gfx[0-9a-z]*'

Step 2 — look it up. Set an override only if the row says to.

Product-to-target mappings are from AMD's ROCm compatibility matrix and the LLVM AMDGPU processor table; the built-target list is whatever your own log prints in the supported types:[...] bracket, which is the only list that is authoritative for the build you are running. Read that bracket before you set anything — it changes between Ollama releases.

Step 3 — set it where the server will actually see it. Exporting it in your shell does nothing for a systemd service:

sudo systemctl edit ollama
# add:
#   [Service]
#   Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"
sudo systemctl daemon-reload && sudo systemctl restart ollama

The value must be exactly three numbers — major.minor.stepping. HSA_OVERRIDE_GFX_VERSION=11.0 is not shorthand for 11.0.0; it is a parse error. On a machine with two different AMD targets, use the per-node form HSA_OVERRIDE_GFX_VERSION_0 / _1 so you do not convert a working card into a broken one. The full per-card table, the parsing rules and the failure modes when an override is wrong are in our ROCm unsupported-GPU and HSA_OVERRIDE_GFX_VERSION guide; the from-scratch install path is in the AMD ROCm local LLM setup guide.

Three AMD failure modes an override will not touch:

1. Group membership. /dev/kfd needs video and/or render — that is cause 5, and on AMD it is the most common one of all.
2. iGPU memory accounting. On APUs, Linux 6.9.9+ changed how GTT memory is reported, and Ollama has historically skipped integrated GPUs whose apparent VRAM is too small. PR #6282 documents the symptom — server hangs and out-of-memory on APUs — and lists the affected targets (gfx1103, gfx1037, gfx1035, gfx1033, gfx1036, gfx1151, gfx1152, gfx940, gfx90c). Note that PR was closed without being merged, so treat it as a description of the problem rather than a shipped fix; check your own version's behaviour. PR #5426 is a community write-up of the 780M path on Linux.
3. A missing ROCm userspace. amdgpu detected, but no compatible rocm library found is not an override problem. Install the ROCm version Ollama's docs ask for, then re-read the log.

If none of that lands, Vulkan is the pragmatic fallback on AMD — it does not care about gfx code objects. That is cause 6 in reverse: instead of forcing CUDA off, you leave Vulkan on.

---

What This Page Does Not Cover

Being straight about the edges of this page. Everything here is sourced from Ollama's documentation, its issue tracker and AMD's published support matrices. We do not own this hardware and there are no first-hand benchmarks on this page — where a number appears, either the source is named or the arithmetic is shown so you can check it.

• No performance figures. Not for Vulkan versus CUDA, not for one card against another. We can tell you how to see which backend was selected and how to force the other one; the tokens-per-second comparison has to be yours, on your prompt.
• The compute-capability table is transcribed, not tested. It comes from Ollama's published hardware docs. If you are on a 5.x or 6.x card, treat the driver-570 requirement as the documented requirement and verify against the docs shipped with your Ollama version.
• The AMD override table is a mapping, not a guarantee. Product-to-gfx mappings come from AMD and LLVM; whether a given override works on your build depends on the supported types:[...] list your own log prints. Where LLVM lists a target's retail names as TBA — gfx1032 and gfx1034 are both in that state — the mapping is the long-established one from rocminfo output, and we have flagged it rather than dressing it up as official.
• Issue-tracker counts are point-in-time. The 152 / 592 / 966 / 1,227 figures are GitHub search results for ollama/ollama on 2026-08-22 and will drift.

If a step here does not match what you see, the version you are running is the first thing to check. ollama --version — and if it is not v0.32.x, expect different log lines.

---

Sources

• Ollama hardware documentation (docs/gpu.mdx) — compute capability floor, driver 550/570 bands, supported card table, Vulkan and ROCm environment variables
• Ollama troubleshooting documentation — log locations, CUDA error codes 3/46/100/999, CUDA_ERROR_LEVEL, OLLAMA_LLM_LIBRARY, Docker cgroup and noexec-tmp fixes
• Ollama FAQ — ollama ps PROCESSOR states, 4096-token default context, OLLAMA_KV_CACHE_TYPE, keep-alive behaviour
• Ollama v0.32.14 release — the version this page was written against (2026-08-15)
• Hugging Face model configs for Qwen3-8B and Qwen3-32B — layer, KV-head and head-dim values used in the KV-cache arithmetic
• AMD ROCm compatibility matrix and the LLVM AMDGPU processor table — the official gfx-target list and product-name mapping behind the cause 8 table
• ollama/ollama tracker threads quoted in cause 8: #738 AMD GPU & ROCm support, #2453 gfx803/gfx802/gfx805, #2637 Integrated AMD GPU support, PR #6282 APU GTT memory, PR #5426 Radeon 780M on Linux — the source of the log-line wording and the affected gfx targets

---

FAQ