MoE VRAM Calculator: Run 30B-120B Models on 8-24GB

Bits Per Parameter, Measured From Real Files

Q4_K_M is not 4 bits per parameter — on Qwen3-30B-A3B it works out to 4.86. K-quants mix precisions per tensor, and MXFP4 releases keep attention in higher precision than the experts. Rather than assume a nominal width, the calculator divides real published file sizes by real parameter counts.

Sizes from the Hugging Face file listings for unsloth/Qwen3-30B-A3B-GGUF, ggml-org/gpt-oss-20b-GGUF and ggml-org/gpt-oss-120b-GGUF on 18 August 2026. More on how the formats differ in our quantization format comparison.

The Two Flags That Do the Work

-ncmoe N is the dial and -cmoe is the sledgehammer. Both are documented in the llama.cpp CLI reference, verbatim: -cmoe, --cpu-moe — “keep all Mixture of Experts (MoE) weights in the CPU”; -ncmoe, --n-cpu-moe N — “keep the Mixture of Experts (MoE) weights of the first N layers in the CPU”. They are current, not folklore: both appear in the reference as of release b10472 (17 August 2026).

# Qwen3-30B-A3B Q4_K_M on a 12GB card
llama-cli -m Qwen3-30B-A3B-Q4_K_M.gguf -ngl all -ncmoe 22 -c 8192

# gpt-oss-20b on a 12GB card — barely any offload needed
llama-cli -m gpt-oss-20b-MXFP4.gguf -ngl all -ncmoe 3 -c 8192

# Very little VRAM: push every expert to the CPU
llama-cli -m Qwen3-30B-A3B-Q4_K_M.gguf -ngl all -cmoe -c 8192

# Measure it properly instead of trusting any calculator
llama-bench -m Qwen3-30B-A3B-Q4_K_M.gguf -ngl 99 --n-cpu-moe 18,22,26,30

Note the pairing: you still pass -ngl all. The point of -ncmoe is that all layers go to the GPU and only the expert tensors within the first N of them are held back — which is why it beats the old approach of simply reducing -ngl. Reducing -ngl strands whole layers, attention included, on the CPU. The equivalent hand-rolled version is an -ot, --override-tensor regex, which is what people used before these flags existed and is still there if you need finer control.

What the Calculator Says for the Common Cards

At 8K context, a 24GB card holds Qwen3-30B-A3B Q4_K_M outright, and a 12GB card runs it with 8 GB of experts in system RAM. These are the calculator's own outputs, reproduced so you can see the shape before you touch a slider.

8K context, fp16 KV cache, 0.7 GB runtime overhead allowance. The gpt-oss-120b row is the honest one: the flag value barely moves with VRAM because the constraint is the ~57 GB of MXFP4 experts you need somewhere. If you do not have 64 GB of system RAM, that model is not a candidate no matter which card you own — see the hardware hub for what actually holds it.

What This Calculator Does Not Know

It computes memory exactly and speed approximately, and you should treat those two outputs very differently.

• The tokens-per-second figures are bandwidth rooflines, not benchmarks. They divide the bytes read per token by the memory bandwidth on each side of the split. They ignore compute, kernel launch overhead, PCIe transfers and prompt processing, so real throughput lands below them. We publish the ceiling because the ratio between offload settings is what you are choosing between; for the absolute number, run llama-bench on your own box.
• Prompt processing behaves differently from decode. During prefill every expert can be touched, so an offloaded model is disproportionately slower on long prompts than the per-token figures suggest. If you feed it 8K-token documents, measure prefill separately.
• The KV cache assumes fp16 and no cache quantization. Quantizing the KV cache changes the fixed cost materially at long context; the calculator does not model it.
• Overhead is a flat 0.7 GB allowance. Real runtime overhead varies by backend, batch size and driver. If you are within a few hundred megabytes of your VRAM limit, the calculator is telling you it is tight — not that it will work.
• Unified-memory machines break the model entirely. On Apple Silicon and Strix Halo there is no VRAM/RAM split to optimise, so -ncmoe is not the lever it is on a discrete card.

When to Use a Different Tool

Use this page only for Mixture-of-Experts models. For a dense model — Llama, Mistral, Gemma, the Qwen3 dense line — the classic parameters-times-bits arithmetic is correct, and the general VRAM calculator is the right tool. If you are still deciding which size class to target, the model size picker works backwards from your hardware, and our 12GB VRAM model picks covers what runs well on the most common card tier. For running any of this through Ollama rather than llama.cpp directly, start with the complete Ollama guide. Model-specific detail lives on the gpt-oss and Qwen3-Coder pages.

Frequently Asked Questions

Sources

• Qwen/Qwen3-30B-A3B config.json — 48 layers, hidden 2048, 128 experts, 8 per token, moe_intermediate_size 768, 4 KV heads, head_dim 128
• openai/gpt-oss-20b config.json and gpt-oss-120b config.json — 24 / 36 layers, hidden 2880, 32 / 128 experts, 4 per token, alternating sliding_attention with a 128-token window
• llama.cpp CLI reference — the -cmoe / --cpu-moe and -ncmoe / --n-cpu-moe N definitions quoted above, release b10472 (17 August 2026)
• Hugging Face file listings for unsloth/Qwen3-30B-A3B-GGUF, ggml-org/gpt-oss-20b-GGUF and ggml-org/gpt-oss-120b-GGUF — quantised file sizes and GGUF parameter counts
• Wikipedia GeForce RTX 30 / 40 / 50 series specification tables — published memory-bandwidth figures used in the roofline estimate