GGUF vs GPTQ vs AWQ: Which Quantization to Use

Short answer: use GGUF unless you have a specific reason not to. GGUF is the only one of the three that runs on CPU, on consumer NVIDIA and AMD GPUs, and on Apple Silicon, and it is the format Ollama, LM Studio and llama.cpp all speak natively. GPTQ and AWQ are CUDA-first server formats — pick them when you are running vLLM or ExLlama on an NVIDIA box and want higher batched throughput than llama.cpp gives you.

The comparison people usually want — "which one is most accurate?" — matters far less than it sounds. All three lose a small amount of quality at 4-bit and the gap between them is smaller than the gap between 4-bit and 5-bit of the same format. What actually decides the choice is which runtime you are using and how much memory you have.

What does quantization actually do?

Model weights are stored as numbers. A model released at FP16 uses 16 bits per weight. Quantization re-encodes those weights at lower precision — commonly 4 to 8 bits — so the file shrinks, and so does the amount of memory bandwidth the GPU has to chew through for every token it generates. That second effect is why quantized models are faster, not just smaller.

The cost is rounding error. Every weight lands on a coarser grid than it was trained on, and the model's output drifts slightly. How much it drifts depends on how cleverly the format handles the outlier weights that matter most.

Key principle: fewer bits means a smaller file, less memory traffic, and faster generation — plus more approximation error. Every quantization format is an attempt to buy the first three while limiting the fourth.

Bit-width arithmetic

This table is pure division against a 16-bit baseline, so you can check it yourself:

One wrinkle: a "4-bit" GGUF file is not exactly 4 bits per weight. K-quants store per-block scale and minimum values alongside the packed weights, which pushes a Q4_K_M file to roughly 4.8 bits per weight in practice. Divide that by 8 bits per byte and you get 0.6 bytes per parameter — the rule of thumb the rest of this page uses.

How much VRAM will a quantized model need?

Here is the formula. It is worth memorising because it answers 90% of "will this fit?" questions:

VRAM for weights (GB) ≈ parameters (billions) × 0.6      [at Q4_K_M]
Total VRAM ≈ weights + KV cache + ~1GB runtime overhead

Applied across the model sizes people actually run:

The multipliers for the other quants come from the same division: Q5_K_M lands near 5.9 bits per weight (~0.74 bytes per parameter), Q8_0 near 8.5 (~1.06 bytes). Multiply by your parameter count and add headroom for context.

Cross-check any of these against our Ollama RAM and VRAM table, which lists the actual published download sizes tag by tag.

Will quantization make my model faster?

Yes, and you can put an upper bound on how fast. Token generation on a single request is memory-bandwidth bound: to emit one token the GPU reads essentially every weight once. So:

throughput ceiling (tokens/sec) = memory bandwidth (GB/s) ÷ model size (GB)

This is an arithmetic upper bound, not a prediction. Real output lands well below it — attention overhead, the KV cache, sampling and framework overhead all take a cut, and a realistic figure is typically somewhere in the region of half to two-thirds of the ceiling. Use it to answer "is this plausible?" rather than "what will I get?"

Worked example, using memory bandwidth figures from the vendors' own specification pages:

The useful consequence: going from Q8 to Q4 roughly halves the bytes read per token, so it roughly doubles the ceiling. That is the real reason quantization feels fast, and it is why a model that just fits in VRAM at a higher quant is usually slower than the same model at a lower one.

GGUF vs GPTQ vs AWQ: what is the actual difference?

The one structural difference worth internalising: GGUF can put some layers on the GPU and the rest in system RAM. GPTQ and AWQ largely cannot. If a model is 4GB bigger than your card, GGUF degrades gracefully and the others simply refuse. On consumer hardware that flexibility outweighs any accuracy difference between the formats.

What about quality?

Both GPTQ and AWQ published perplexity and downstream-task comparisons in their original papers, linked in the table above, and those are the numbers worth reading — they include the calibration setup and the baselines, which any second-hand summary strips out. The llama.cpp repository tracks the same thing for GGUF k-quants across releases.

The honest summary across all three: at 4-bit the degradation on a 7B-and-up model is small but real, and it shows up first on long-chain reasoning and code rather than on chat. At 8-bit it is close enough to FP16 that most people cannot tell. We have not run an independent evaluation, so we are not going to publish a leaderboard that implies we did — if you need a number for a specific model, the model card and the format's own paper are the primary sources.

Which format should I pick?

Work down this list and stop at the first line that matches you:

1. You use Ollama, LM Studio, or Jan. → GGUF. It is the only format those tools load.
2. You are on a Mac, or have no discrete GPU. → GGUF. Nothing else runs on Metal or CPU.
3. The model is bigger than your VRAM. → GGUF, so you can offload the overflow to system RAM.
4. You run vLLM and serve several users at once. → AWQ. That is the combination it is designed and packaged for.
5. You run ExLlamaV2 or text-generation-webui on NVIDIA and want maximum single-stream speed. → GPTQ.
6. None of the above. → GGUF Q4_K_M, and move on to something that matters more.

Picking a GGUF quant level

Given that GGUF is the answer for most people, the follow-up question is which of its dozen tags to pull:

If a Q4_K_M model fits your card with several GB to spare, step up to Q5_K_M or Q6_K rather than jumping to a bigger model at Q3 — a smaller model at a higher quant is usually the better output.

How do I test a quantized model on my own hardware?

Nobody else's benchmark is a substitute for ten minutes on your own machine and your own prompts. The procedure:

# Pull two quant levels of the same model
ollama pull llama3.1:8b-instruct-q4_K_M
ollama pull llama3.1:8b-instruct-q8_0

# Run with --verbose to get eval rate and load time
ollama run llama3.1:8b-instruct-q4_K_M --verbose

# Check what actually landed on the GPU
ollama ps

ollama ps is the important one: the PROCESSOR column tells you whether the model is 100% GPU or has spilled to CPU, and a partial spill explains almost every "why is this so slow" question.

Then score the same ten prompts across both quants. Use prompts from your real workload, not trivia — a hard refactor, a long summarisation, a multi-step reasoning question. Record eval rate from --verbose, peak memory from ollama ps, and a subjective 1-5 quality mark. Ten minutes of that beats any published table, because it is measured on the machine you will actually use.

Frequently asked questions

Which quantization should I use for daily chat on 8-16GB hardware? GGUF Q4_K_M. It is the only format that runs on CPU and Apple Silicon, it is what Ollama and LM Studio load by default, and at ~0.6 GB per billion parameters a 7B or 8B model leaves room for a decent context window on an 8GB card.

Does GPTQ still matter? Yes, in one lane: CUDA-only inference where you are already running ExLlamaV2 or text-generation-webui and want maximum single-stream throughput. Outside that lane it is a worse GGUF — it needs a calibration dataset, it cannot offload to CPU, and it does not run on Apple or AMD hardware.

When should I choose AWQ over GGUF? When you are serving with vLLM. AWQ is the format vLLM's ecosystem packages and documents most thoroughly for 4-bit weights, and vLLM's batching is where it earns its keep. For a single user on a desktop, AWQ gives you nothing GGUF does not.

How much quality do I actually lose at 4-bit? Enough to notice on long code and multi-step reasoning, not enough to notice in chat. The published comparisons in the GPTQ and AWQ papers are the right primary sources; both report perplexity against FP16 baselines with their calibration setups stated.

What hardware works best with each format? CPU-only and Apple Silicon: GGUF, no alternative. Consumer NVIDIA (3060 through 4090): GGUF for flexibility, GPTQ if you want ExLlamaV2 speed. AMD: GGUF via Vulkan or ROCm; AWQ works through vLLM on ROCm but the path is rougher. Multi-GPU servers: AWQ or GPTQ under vLLM.

Is a bigger model at Q3 better than a smaller model at Q6? Usually the bigger model wins down to about Q4, and usually loses below it. A 32B at Q4 beats a 13B at Q8. A 32B at Q2 is a coin flip against a 13B at Q5. Q4 is the floor where the trade still reliably favours parameters.

What is the difference between per-tensor and per-channel quantization? Per-tensor uses one scale factor for an entire tensor; per-channel keeps a separate scale per output channel. Per-channel costs slightly more storage and preserves more detail, particularly in attention weights. GGUF k-quants go further still with per-block scales inside each tensor, which is exactly why a Q4_K file runs closer to 4.8 bits per weight than to 4.

Why does the calibration dataset matter for GPTQ and AWQ? Both formats decide which weights to protect by observing activations on sample data. If that sample looks nothing like your workload, the format protects the wrong weights. GGUF sidesteps this entirely by default — no calibration pass, which is one reason the same GGUF file behaves consistently across very different uses.

Next steps

• Work out what your card can hold: best Ollama models for 8GB VRAM and the full RAM/VRAM table.
• Running light: best local AI models for 8GB RAM.
• Something not loading: troubleshooting local AI covers the VRAM and driver failures quantization causes.
• Starting from zero: install your first local AI.