GGUF vs FP8 vs Nunchaku INT4: Which FLUX Quant Fits

Short answer, by card: 8GB → Q4_K_S GGUF (6.81 GB) or Nunchaku INT4 (6.77 GB). 12GB → Q6_K GGUF (9.86 GB). 16GB+ → Q8_0 GGUF (12.7 GB), or flux1-dev-fp8.safetensors (17.2 GB) if you want one file instead of four. Nunchaku is the speed play, not the disk-space play — its INT4 file is the same size as a Q4 GGUF. All sizes below are read from the Hugging Face repos, not estimated.

You are staring at a repo listing with four files that are supposedly the same model, and the sizes range from 4 GB to 24 GB. Nobody tells you what the difference costs. Here is the whole decision, with the numbers, plus the two install traps that eat an evening if you pick Nunchaku without reading first.

---

The Four Files, Measured

Same 12B FLUX.1-dev transformer, five ways to store it — a 5.9x spread from Q2_K to F16. These are exact repo file sizes as of August 18, 2026:

Two things jump out immediately.

First: there is no Q4_K_M in city96's FLUX.1-dev repo. People search for it constantly because that is the default recommendation on the LLM side, and they end up on a page that does not have it. The nearest files are Q4_K_S (6.81 GB) and Q4_0 (6.79 GB). Take Q4_K_S. (The newer FLUX.2 klein GGUF sets do ship Q4_K_M — see below.)

Second: Nunchaku INT4 is not smaller than a Q4 GGUF. 6.77 GB versus 6.81 GB is a rounding error. If you came to Nunchaku expecting a tiny download, recalibrate now — SVDQuant carries a 16-bit low-rank branch alongside the 4-bit weights, and that branch is where the file size goes. What you buy is throughput and a low runtime floor, not disk.

---

What Each Format Actually Is

GGUF is flexible and universally loadable, fp8 is one setting that just works, Nunchaku is a whole inference engine that happens to ship 4-bit weights. They are not three grades of the same thing.

GGUF — the llama.cpp container format, brought to diffusion models by city96/ComfyUI-GGUF (3.9k stars). You install the custom node, drop the file in ComfyUI/models/unet, and load it with the Unet Loader (GGUF) node (it lives in the bootleg category, which throws people the first time). The appeal is granularity: ten quantization steps for FLUX.1-dev (Q2_K through Q8_0) plus an F16 file, so you can dial the file to your card instead of the other way round. If you have used GGUF for text models, the naming carries over — our quantization explainer covers what the K-quants mean, though it is written for LLMs.

fp8 — 8-bit floating point, natively supported by PyTorch on Ada (RTX 40) and newer, with software emulation elsewhere. There is essentially one fp8 file per model, so there is no decision to make. Comfy-Org's flux1-dev-fp8.safetensors is the canonical one for ComfyUI.

Nunchaku / SVDQuant — nunchaku-tech/nunchaku, 3,933 stars, an MIT-HAN-Lab inference engine implementing 4-bit weights and activations. Its own README claims 3.6x memory reduction compared to the BF16 model, an INT4 model that runs 3x faster than the NF4 W4A16 baseline, and an 8.7x speedup over the 16-bit model on a 16GB laptop 4090 once CPU offloading is eliminated. Those are the project's own benchmark figures, published with an efficiency chart — quotable, but not independently reproduced here. The README also claims that a 4-bit text encoder plus per-layer CPU offloading brings FLUX's minimum memory requirement down to 4 GiB while maintaining a 2-3x speedup.

---

The Download-Size Trap

The GGUF file is not the whole download. Budget roughly 12 GB total for a Q4 GGUF setup, not 6.8 GB.

The bf16 and GGUF FLUX files contain the transformer only. FLUX also needs two text encoders and a VAE, and you download those separately:

Sizes from comfyanonymous/flux_text_encoders, city96/t5-v1_1-xxl-encoder-gguf and black-forest-labs/FLUX.1-dev.

So a Q4_K_S GGUF stack is 6.81 + 4.89 + 0.25 + 0.34 ≈ 12.3 GB on disk. On an 8GB card that still works, because ComfyUI runs the text encoders first and unloads them before the transformer runs — but if you are tight on VRAM and on RAM, the Q4_K_S T5 encoder at 2.74 GB is the lever most people never touch.

The fp8 checkpoint is the mirror image: bigger file, fewer of them. Comfy-Org's model card says outright that "the two text encoders used by Flux are already included in this one safetensor," which is why 17.2 GB is more than the ~12 GB you would expect from 12B parameters at a byte each.

And the trap inside the trap: that file belongs in ComfyUI/models/checkpoints/ and loads with Load Checkpoint. The GGUF and bf16 transformer-only files belong in unet/ or diffusion_models/ and load with a UNET/Unet Loader. Putting the all-in-one checkpoint behind a UNET loader (or vice versa) produces confusing shape errors that look like a corrupt download. If you are new to the folder layout, our ComfyUI complete guide walks it end to end.

---

Pick by VRAM

If you want one line: match the transformer file to about 80% of your VRAM and let ComfyUI swap the encoders around it.

Card-by-card capacity for every FLUX variant lives on our FLUX VRAM requirements by GPU page, and if you are still fighting to get anything to generate at all, start with run FLUX on a low-VRAM GPU. For the model picks that go with each tier: 8GB, 12GB and 16GB.

---

The Nunchaku Install Tax

Nunchaku is not a file you download — it is a compiled wheel that must match your Python, PyTorch and CUDA versions exactly. This is where most people give up, so here is the whole matrix before you start.

The v1.2.1 release (25 January 2026) ships prebuilt wheels across this grid:

If your environment is outside that grid — PyTorch 2.7, Python 3.9, an ARM Linux box — there is no wheel and you are building from source. Wheel filenames encode all of it, so pick yours literally:

# Example shape — substitute YOUR cuda/torch/python versions
pip install https://github.com/nunchaku-tech/nunchaku/releases/download/v1.2.1/nunchaku-1.2.1+cu12.8torch2.9-cp311-cp311-linux_x86_64.whl

Check what you actually have first:

python --version
python -c "import torch; print(torch.__version__, torch.version.cuda)"

ComfyUI portable users: the wheel must go into ComfyUI's bundled Python, not your system Python. Launch ComfyUI, read the console line that starts ** Python executable:, and install with that exact path. The ComfyUI-nunchaku node pack (2.9k stars) also ships an install_wheel.json workflow that does this for you from inside ComfyUI.

GPU architecture support, per the official install docs, is an explicit list:

Anything not on that list — Pascal, most Intel and AMD cards, Apple Silicon — is out. Nunchaku is CUDA-only.

Then pick the right precision. The model card is unambiguous: INT4 is "for users with non-Blackwell GPUs (pre-50-series)", NVFP4 is "for users with Blackwell GPUs (50-series)." Blackwell owners also need PyTorch 2.7+ with CUDA 12.8+.

LoRAs are fine. The docs say Nunchaku "seamlessly integrates with off-the-shelf LoRAs without requiring requantization" — the LoRA is concatenated onto the low-rank branch rather than fused into 4-bit weights, so no conversion step. The one documented gotcha is multi-LoRA: set_lora_strength applies a single uniform strength across every loaded LoRA, so if you need different weights per LoRA you must pass per-LoRA strengths into compose_lora instead.

One honest flag on project health. Nunchaku's last release is v1.2.1 on 25 January 2026, and the last push to the main repo was 7 March 2026. As of 18 August 2026 that is roughly five months of quiet. The code works and the wheels cover recent PyTorch builds — but if you are picking a format to build a pipeline on for the next year, factor that in against GGUF, whose loader tracks ComfyUI actively.

---

What We Could Not Measure

We are not going to show you a side-by-side quality grid we did not generate. Doing this properly means one card, one prompt, one seed, one step count, and peak-VRAM plus seconds-per-image logged for every format — and publishing the images so the quality claim is visible rather than asserted. We have not run that on a controlled rig, so this page states file sizes, documented behaviour and vendor claims, clearly labelled, and stops there.

What that means for you, practically:

• Speed claims are Nunchaku's own. 3x over NF4, 8.7x on a 16GB laptop 4090, 3.6x memory reduction — all from the project README and its efficiency chart. Reproduce them on your card before planning around them.
• The "4 GiB minimum" figure is a project claim too, and it depends on CPU offloading, which trades VRAM for wall-clock time. Low VRAM does not mean fast.
• Quality ranking below Q4 is genuinely subjective. The consistent community pattern is that Q8_0 is visually hard to separate from bf16, Q6_K is close, and degradation becomes obvious somewhere around Q3/Q2 — with fine text, hands and small faces failing first. Treat that as folklore until you have compared your own prompts.

If you want a starting protocol: fix the seed, generate the same prompt at bf16 and at your candidate quant, and look at text rendering and hands before anything else. Those break first.

---

FLUX.2 Changes the Map

If you are shopping today, you may not want FLUX.1 at all — and FLUX.2 klein has a different, cleaner format story. Black Forest Labs ships official fp8 builds, and the GGUF sets are more complete than FLUX.1-dev's:

Sizes from black-forest-labs/FLUX.2-klein-{4b,9b}-fp8 and unsloth/FLUX.2-klein-{4B,9B}-GGUF. Note that the klein fp8 files are transformer-only — unlike the FLUX.1 all-in-one checkpoint — and that klein 4B at Q4_K_M is 2.60 GB, which puts real FLUX-family quality inside a 6GB card for the first time.

Nunchaku does not officially support FLUX.2. Its README model list stops at FLUX.1-dev, FLUX.1-Tools, FLUX.1-Kontext, FLUX.1-Krea-dev, Qwen-Image, Qwen-Image-Edit, Qwen-Image-Edit-2509, Z-Image-Turbo, SANA and PixArt-Sigma, and there is no FLUX.2 repo under the nunchaku-ai Hugging Face account. Community conversions exist — tonera/FLUX.2-klein-9B-Nunchaku is the one you will find — but they are unofficial and unsupported. If FLUX.2 is your target, the choice is fp8 or GGUF.

---

Verdict

1. Default to GGUF. Ten size steps from Q2_K to Q8_0, one custom node, works on NVIDIA, AMD and Apple Silicon alike, and the loader is actively maintained. Q4_K_S at 8GB, Q6_K at 12GB, Q8_0 at 16GB.
2. Take fp8 if you value one file over one gigabyte. At 16GB+, flux1-dev-fp8.safetensors removes four downloads and two folder decisions. It costs you 4.5 GB versus Q8_0 and gives you no way to go smaller.
3. Nunchaku is a speed upgrade with an install bill. Right GPU architecture, right precision build, right wheel for your exact Python/torch/CUDA. Clear those and the project claims 2-3x with offloading, more without. Do not choose it expecting a smaller download — 6.77 GB versus 6.81 GB is nothing.
4. Check the project pulse before committing. Nunchaku has been quiet since March 2026 and has no official FLUX.2 support. GGUF has neither problem.
5. Do not download bf16 unless you are benchmarking. 23.8 GB to produce images most people cannot distinguish from a 12.7 GB Q8_0.

Everything on this page is free and runs on hardware you already own. If you are still deciding what to own, our hardware hub has the current buying map.

---

Sources

• nunchaku-tech/nunchaku — README claims, release wheels, installation docs, LoRA usage docs
• nunchaku-ai/nunchaku-flux.1-dev — INT4/NVFP4 file sizes and the per-architecture precision guidance
• city96/FLUX.1-dev-gguf and city96/ComfyUI-GGUF — GGUF file sizes and the loader node
• Comfy-Org/flux1-dev — fp8 checkpoint size and the "two text encoders included" statement
• comfyanonymous/flux_text_encoders, city96/t5-v1_1-xxl-encoder-gguf, black-forest-labs/FLUX.1-dev — encoder and VAE sizes
• black-forest-labs/FLUX.2-klein-9b-fp8 and unsloth/FLUX.2-klein-9B-GGUF — FLUX.2 klein file sizes

All file sizes and repository statistics read on August 18, 2026.

---

FAQ