NPU vs GPU vs CPU for Local AI (2026): Copilot+ or RTX

For running local LLMs in 2026, a discrete RTX GPU beats a Copilot+ PC's NPU, and the gap is not close. The four silicon classes split cleanly: the NPU is a fixed-function, inference-only block for small always-on models at single-digit watts; the integrated GPU and the CPU share one system memory bus and are what your local LLM actually runs on today; the discrete GPU is the only one with a private, very wide memory bus, and that is the thing that decides tokens per second. Copilot+ NPUs are real accelerators (Snapdragon X at 45 TOPS, Snapdragon X2 Elite at 80, Intel Lunar Lake at 48, AMD Ryzen AI 300 at 50), but Ollama and llama.cpp do not target them, and no NPU trains anything — training happens elsewhere and the finished model is converted for deployment. Buy a Copilot+ PC for all-day battery and built-in Windows AI; buy or build an RTX machine if running 7B-70B local models at speed is the actual goal.

This guide compares the four silicon classes inside a modern machine — NPU, integrated GPU, discrete GPU and CPU — for one specific job: running local large language models. Every number below is either a vendor-published spec with the vendor named, or arithmetic with the working shown.

NPU vs GPU vs CPU: what each class is actually for

This is the table most comparisons skip. The four blocks are not competing implementations of the same idea — they are built for different jobs, and the differences that matter for local AI are the memory model and the power envelope, not the marketing TOPS figure.

Two rows do the real work here. Only the discrete GPU has a private memory bus — everything else in a laptop is drinking from the same LPDDR5X straw. And only the NPU is fixed-function: it does not execute arbitrary code, which is why it needs a vendor toolchain, and why it is the one class that cannot train.

What is a Copilot+ PC, and what is its NPU?

A Copilot+ PC is Microsoft's certification for Windows 11 laptops with a dedicated NPU (Neural Processing Unit). Microsoft's Copilot+ PCs developer guide sets the bar at an NPU "that can perform more than 40 trillion operations per second (TOPS)", paired with at least 16GB of RAM and 256GB of storage, and describes the NPU as "a specialized computer chip for AI-intensive processes like real-time translations and image generation".

As of mid-2026 the main Copilot+ chips, on vendor-published ratings:

• Qualcomm Snapdragon X Elite / X Plus — Arm-based, 45 TOPS Hexagon NPU, 135 GB/s LPDDR5X.
• Qualcomm Snapdragon X2 Elite — the 2026 refresh, 80 TOPS (up to 85 on the top Elite Extreme SKUs), LPDDR5X up to 228 GB/s and up to 128GB.
• Intel Core Ultra 200V "Lunar Lake" — x86, 48 TOPS NPU (Intel quotes its figure at INT8).
• AMD Ryzen AI 300 "Strix Point" — x86, 50 TOPS XDNA 2 NPU (INT8).

One caveat before you compare those: Qualcomm publishes no precision qualifier on its TOPS numbers, so setting a Qualcomm figure against an Intel INT8 figure is an assumption, not an equivalence. The full per-SKU breakdown with each vendor's own labelling is in our NPU TOPS ranking table for every CPU with an NPU.

That TOPS number sounds enormous next to a phone, but TOPS measures peak low-precision throughput on small models. It is not the metric that decides how fast a multi-billion-parameter LLM generates text. The thing that decides that is memory bandwidth.

NPU vs discrete GPU: spec-for-spec

A Copilot+ NPU and a discrete RTX GPU optimise for opposite ends of the same trade-off. The NPU optimises for performance per watt; the GPU optimises for raw parallel throughput and bandwidth. For LLM token generation, bandwidth is king.

The row that decides everything is memory bandwidth. A Copilot+ laptop shares one pool of LPDDR5X between CPU, iGPU and NPU: 135 GB/s on the first-generation Snapdragon X Elite per Qualcomm's product page, up to 228 GB/s on the 2026 Snapdragon X2 Elite. A discrete RTX card has its own 24GB at 936-1,008 GB/s. That is roughly 4x more bandwidth than an X2 Elite and nearly 7x more than an X Elite (936 ÷ 228 ≈ 4.1; 936 ÷ 135 ≈ 6.9). Because LLM text generation streams the model's weights through memory for every single token, that bandwidth gap translates almost directly into a throughput gap.

Our GPU memory bandwidth table for local LLMs lists the same figure for every consumer card if you want to check a specific model.

Can an NPU actually run a local LLM today?

With the normal tools, no. As of mid-2026, mainstream local LLM tooling does not target the NPU at all:

• Ollama has Arm64 Windows builds for Snapdragon, but NPU acceleration is not exposed — on a Snapdragon X Elite everything runs CPU-only through the Ollama CLI.
• llama.cpp runs on these machines and leans on the CPU or the iGPU. Production-grade NPU offload is not there for mainstream use.
• Intel is furthest along: IPEX-LLM can target the Lunar Lake NPU and integrates with llama.cpp and Ollama on Windows, but it is a separate stack, model-limited, and far from plug-and-play.
• Qualcomm's QNN runtime can push a model onto the Hexagon NPU — but through Qualcomm-converted models from Qualcomm AI Hub, not ollama run.

There is a structural reason this is hard, and Microsoft states it plainly in the developer guide: "AI models are often trained and available in larger data formats, such as FP32. Many NPU devices, however, only support integer math in lower bit format, such as INT8, for increased performance and power efficiency. Therefore, AI models need to be converted (or 'quantized') to run on the NPU." A model has to be compiled for the NPU. A GPU simply executes it.

So when you "run a local model" on a Copilot+ PC with the normal tools, you are running it on the CPU — see running LLMs on CPU only for what that path realistically delivers. The NPU is busy doing what it was designed for: Windows Studio Effects, live captions and Recall, not 8-billion-parameter chatbots. The same split exists on Apple silicon, where Ollama uses Metal rather than the Apple Neural Engine.

Are NPUs used for LLM training?

No. Every consumer NPU shipping in 2026 — Qualcomm Hexagon, AMD XDNA 2, Intel's NPU, Apple's Neural Engine — is an inference-only block. Nobody trains a language model on one, no vendor offers a way to, and no amount of TOPS changes that. Training happens on datacentre accelerators; the finished model is then converted and deployed to the NPU.

Three concrete reasons, each from a vendor's own documentation or from arithmetic you can check yourself:

1. The numeric precision is wrong. Microsoft's guide is explicit that many NPUs "only support integer math in lower bit format, such as INT8", and that models must therefore be quantized before they will run at all. Training needs high-precision gradients and optimizer state; a fixed-function INT8 array has no way to represent them.
2. The toolchain is deploy-only by design. AMD's Ryzen AI software documentation describes a flow that "does not require any modifications to the existing model training processes" — you train somewhere else, then convert the finished model for the NPU. Qualcomm's route has the same shape: pre-validated models arrive through Qualcomm AI Hub. Neither vendor ships a training API for the NPU, because there is no training silicon to expose.
3. The memory is nowhere near enough. Full fine-tuning in mixed precision holds several copies of the model at once. Count the bytes per parameter: 2 (fp16 weights) + 2 (gradients) + 4 (fp32 master copy) + 8 (Adam's two moment estimates) = 16 bytes per parameter. A 7B model is therefore 7 × 16 = 112GB before activations or batch data — on a laptop whose entire 16-128GB pool is already shared with Windows.

What you can do on a Copilot+ machine is parameter-efficient fine-tuning (LoRA or QLoRA) of a small model — but that runs on the CPU or the iGPU, not on the NPU, and it is slow. If fine-tuning is the goal, the answer is a discrete GPU, and our best GPUs for local AI guide ranks them by VRAM and value.

Why does the GPU win so decisively on tokens per second?

LLM inference has two phases, and they are bottlenecked by completely different things:

1. Prefill (prompt processing) — the whole prompt is read in parallel. This is compute-bound, and it is the one phase where a high-TOPS NPU can look good.
2. Decode (generating the answer) — tokens come out one at a time, and each token requires streaming all of the model's active weights out of memory. This is memory-bandwidth-bound.

Almost all of the time you feel in a chat is decode, and decode has a hard ceiling you can compute yourself: memory bandwidth ÷ model size in memory = maximum tokens per second. A Q4_K_M model occupies roughly 0.6GB per billion parameters, so an 8B model is about 8 × 0.6 = 4.8GB of weights. Divide:

These are ceilings, not measurements. Nothing reaches its theoretical bandwidth limit, and the CPU path on a Copilot+ machine lands a long way under its own ceiling, because a handful of CPU cores cannot saturate the bus the way thousands of GPU cores can. But the ceilings alone settle the argument: the NPU's TOPS rating does not appear anywhere in that arithmetic, and doubling it from 45 to 85 does not move the ceiling by a single token. That is why a $1,800 Copilot+ laptop can lose badly to a used 24GB card at the one job of running a chatbot fast.

Does Arm vs x86 change local AI performance?

Less than people expect, because the bottleneck is the same on both. Snapdragon X and X2 are Arm; Lunar Lake and Ryzen AI 300 are x86. All four share a single system memory pool, so all four hit the same bandwidth-shaped ceiling on decode.

Where the instruction set genuinely matters is software, not speed:

• On Arm, x64 Windows applications run under emulation. That has improved a great deal, but it is still why some AI tooling — CUDA-adjacent utilities, older Python wheels, vendor SDKs — is awkward or unavailable. Native Arm64 builds of Ollama and llama.cpp exist, so the core path works.
• On x86, everything in the local AI ecosystem is a first-class citizen, and Intel's IPEX-LLM is the only NPU-offload story that plugs into mainstream tools at all.
• On either, the discrete GPU question is unchanged: with no discrete GPU in the machine, you are running on the CPU or iGPU regardless of instruction set.

The practical read: pick Arm for battery and silence, x86 for compatibility, and neither for LLM speed. The honest guide to local AI on a laptop covers what each class of machine really delivers.

How does NPU performance per watt compare to CPU and GPU?

This is where the NPU genuinely wins, and it is the entire reason the block exists. Microsoft's developer guide states that the Snapdragon X Elite NPU is "using energy on AI tasks more efficiently than a CPU or GPU resulting in longer device battery life", and that Copilot+ features designed for the NPU give users "improved battery life and faster inference execution time".

A word of caution on the numbers you will see quoted around this: no vendor publishes a like-for-like perf-per-watt figure for the same model running on the NPU, the CPU and the GPU of the same machine. Any specific multiplier you find online — "3x more efficient", "half the power" — is almost certainly extrapolated from a vendor slide about a different workload. What is reliably true, and what all the vendor documentation agrees on, is the direction: a fixed-function block running a quantized model draws meaningfully less power than doing the same work on general-purpose cores, and it can do it continuously.

That is why Copilot+ laptops have such good battery life. An RTX laptop GPU under LLM load can pull 100-175W and drain a battery in well under an hour; an NPU doing its intended lightweight work barely registers on the power budget.

The honest framing: NPU and discrete GPU are not really competitors. The NPU is for efficient, always-on, small AI. The discrete GPU is for fast, heavy, on-demand AI like running 7B-70B LLMs. For the full cross-vendor NPU picture, see our best NPU for AI comparison.

So which should you actually buy for local AI?

Match the hardware to what you really plan to do:

• You mainly want to run local LLMs (7B-32B) fast — chat, coding, RAG, agents. Get a discrete RTX GPU. A used RTX 3090 (24GB, 936 GB/s) is the value pick: its bandwidth ceiling on an 8B model is roughly 7x a Snapdragon X Elite's and 4x an X2 Elite's, and it holds far bigger models. A Copilot+ PC will frustrate you here.
• You want a thin, light, all-day laptop with built-in AI features and occasional small-model dabbling. A Copilot+ PC is great — outstanding battery, silent, and the NPU shines on Windows' on-device AI. Just keep LLM expectations to 3B-8B models at modest speeds, running on the CPU.
• You want portability and real local-LLM speed. Today that means a discrete-GPU laptop, an eGPU, or a desktop you reach remotely, accepting worse battery — or a high-memory unified-memory machine, where capacity and bandwidth arrive together.
• You want to fine-tune. Discrete GPU, no exceptions. The NPU cannot train, and the 16-bytes-per-parameter arithmetic above shows why a shared laptop memory pool cannot either.
• You already own a Copilot+ PC and want to try local AI anyway. You can — install Ollama and run a 3B-8B model on the CPU. It works, it is just slow, and you are not using the NPU.

The blunt summary: Copilot+ NPUs are a battery story, not a local-LLM-speed story.

Key Takeaways

1. Four classes, four jobs. NPU = fixed-function, inference-only, single-digit watts, shared memory. iGPU = general parallel compute, shared memory. Discrete GPU = parallel compute with a private wide bus. CPU = flexible, and it is what actually runs your model on a Copilot+ machine.
2. NPUs never train. They support only low-precision integer math (Microsoft's own wording), their toolchains are deploy-only by design (AMD's own wording), and full fine-tuning needs ~16 bytes per parameter — 112GB for a 7B model. Train on a GPU; deploy to an NPU.
3. Most local LLM tools don't use the NPU. On a Snapdragon X Elite, Ollama runs CPU-only. Intel's IPEX-LLM is the closest thing to mainstream NPU offload, and it is a separate, model-limited path.
4. Bandwidth decides token speed, not TOPS. 135 GB/s (X Elite) and up to 228 GB/s (X2 Elite) shared, versus 936-1,008 GB/s private on an RTX card. On an 8B Q4_K_M model (~4.8GB) that is a ceiling of ~28-48 tok/s versus ~195-210 tok/s.
5. Arm vs x86 barely matters for speed. Both share one memory pool. It matters for software compatibility.
6. Recommendation: discrete RTX GPU to run or fine-tune local LLMs; Copilot+ PC for portability, battery and built-in Windows AI. They solve opposite problems.

Next Steps

• Pick the right card with our full ranked guide: Best GPUs for Local AI, from the RTX 3060 up through the latest flagships.
• Check the published bandwidth of any card in the GPU memory bandwidth table for local LLMs and run the ceiling arithmetic yourself.
• Compare every NPU on vendor-published ratings in Best NPU for AI 2026, or look up a single chip in the CPU-with-NPU TOPS ranking table.
• See why the value champion holds up in RTX 3090 for local AI — the best 24GB GPU per dollar for running local models.
• Decided on the Copilot+ side anyway? Our NPU laptop table ranks the machines you can actually buy by memory ceiling and vendor-published bandwidth, not by TOPS.
• Wondering what the NPU in a machine you already own is doing? What uses your NPU on Windows is the app-by-app list, with how to verify each one.
• Want truly portable local AI? Read how to run an LLM on your phone — where mobile NPUs and small models actually shine.
• Not sure which GPU fits your models and budget? Use our Which GPU to buy interactive picker.

Primary sources for the specs above: Microsoft's Copilot+ PCs developer guide for the 40 TOPS requirement, the NPU precision limits and the Windows ML inference path; Qualcomm's Snapdragon X Elite and Snapdragon X2 Elite product pages for TOPS and memory bandwidth; and AMD's Ryzen AI software documentation for the deploy-only NPU flow. To measure your own machine rather than trust anyone's table, the open-source llama.cpp project ships a benchmark that will exercise the CPU, GPU and — where supported — NPU paths on the hardware in front of you.