Can Your GPU Run Qwen3.8-27B? VRAM by Quant

Fit table by VRAM tier

24GB is the intended home; 16GB is the cheapest tier that still feels like the real model. Sizes below are the published file sizes in unsloth/Qwen3.8-27B-GGUF, converted to GiB so they compare directly against your card. Weights are not the whole story — leave headroom for KV cache, and add 0.86 GiB if you want image input.

Cross-check against Unsloth’s own published requirement bands for this model — 11-13 GB at 2-bit, 13-16 GB at 3-bit, 17-19 GB at 4-bit, 24 GB at 6-bit, 31 GB at 8-bit, 56 GB at BF16, measured as total RAM + VRAM. Those are a little higher than raw weights because they include runtime overhead — they are the numbers to plan against, since raw file size never accounts for the runtime.

Not sure which tier you are in? The VRAM calculator and the full Ollama RAM/VRAM table cover the rest of the catalogue.

Every quant, with real file sizes

Twenty-two weight builds exist, from 9.01 GB to a 54.66 GB BF16 split, plus two vision-projector files. These are the actual file sizes in the Unsloth GGUF repository as of 18 August 2026, not derived estimates. The UD- prefix marks Unsloth’s dynamic quants, which vary bit-width per layer instead of applying one setting everywhere.

Ollama’s own tag list is much shorter: twelve tags, all 27B, all reporting a 256K context window and text-plus-image input. The sizes it reports are 18GB (default, q4_K_M, MTP and NVFP4 variants), 30GB (q8_0), 32GB (mxfp8) and 56GB (bf16). If you want anything below 18GB you have to leave Ollama’s registry and pull a GGUF directly.

There is no small Qwen3.8, and that is the whole problem

Qwen published four Qwen3.8 repositories: the 27B, its FP8 twin, a 2.4T-A95B MoE, and that MoE’s FP8 twin. Nothing else. Previous generations trained people to expect a ladder — a 4B, an 8B, a 14B, something that fits a laptop. This generation skipped it. Either you clear roughly 17GB of memory or you are not running Qwen3.8 at anything like full quality.

That vacuum has been filled by community repos, and you should be careful with them. There is an empero-ai/Qwen3.8-9B on Hugging Face, created on 15 August, ten days after Qwen’s own repo went up. It is somebody’s prune of the 27B, not a Qwen model, and no third-party evaluation of it exists. The benchmark table on the official card tells you nothing about it. Treat every “Qwen3.8-<small number>B” repo as an unvalidated experiment.

If 27B does not fit, the honest move is a different family rather than a damaged Qwen3.8. Our local model guide and the 12GB VRAM picks list models actually designed for the memory you have.

The 12GB workaround, and whether it is worth doing

It exists, people are using it, and we would still tell you not to. Someone built and published soyaakinohara/qwen3.8-27b-abliterated-3.69bpw-12GB-MTP.gguf — a single ~12.6 GB file, 3.69 bits per weight, named after the constraint it was built for. It had about 2,300 downloads and 41 likes when we checked on 18 August, so the demand is real.

Two things about it that the filename does not tell you:

• 1.12.6 GB does not fit in a 12GB card. 12 GiB is 12.88 decimal GB, so the weights alone leave roughly 0.28 GB of headroom — less than any usable KV cache. You will be offloading layers to system RAM regardless.
• 2.It is an abliterated build. That means the refusal behaviour has been surgically removed. Whatever you think of that as a feature, it is a modification to the weights on top of a heavy quantisation, and it is not the model Qwen evaluated.

If you genuinely want to see Qwen3.8 on a 12GB card, start with the stock UD-Q2_K_XL at 9.95 GiB instead — same tier of quality loss, no unrelated modifications, and it actually fits. Then be honest with yourself about the output. A 27B model at 2 bits is not obviously better than a well-chosen 12B at 4 bits, and it is considerably slower.

Install: Ollama, LM Studio, llama.cpp

Ollama is one command and gives you the 18GB build; llama.cpp is the only route to the smaller quants.

Ollama

# default — 18GB, q4_K_M, 256K context, text + image
ollama pull qwen3.8:27b
ollama run qwen3.8:27b

# explicit alternatives from the same registry
ollama pull qwen3.8:27b-q8_0     # 30GB
ollama pull qwen3.8:27b-mlx      # 18GB, Apple Silicon MLX build
ollama pull qwen3.8:27b-bf16     # 56GB

LM Studio

The model is in LM Studio’s catalogue as qwen/qwen3.8-27b. Search “qwen3.8” in the Discover tab, or from the CLI:

lms get qwen/qwen3.8-27b

If the model appears but refuses to load, update your llama.cpp runtime in LM Studio’s runtime settings — Qwen3.8 uses the Qwen3.5-family architecture identifier, and older bundled runtimes will not recognise it.

llama.cpp (the only way to get 2-bit and 3-bit)

pip install -U "huggingface_hub[cli]"

# 24GB card
hf download unsloth/Qwen3.8-27B-GGUF \
  --local-dir Qwen3.8-27B-GGUF --include "*UD-Q4_K_XL*"

# 16GB card
hf download unsloth/Qwen3.8-27B-GGUF \
  --local-dir Qwen3.8-27B-GGUF --include "*UD-Q3_K_XL*"

./llama.cpp/llama-cli \
  --model Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_XL.gguf \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0

For image input build the llama-mtmd-cli target as well and pass the mmproj-F16.gguf file alongside the weights. New to llama.cpp or Ollama? Start with the complete Ollama guide.

Sampling settings that matter

Qwen publishes two distinct parameter sets and the difference is not cosmetic. Thinking mode: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0. Non-thinking mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5. Using thinking-mode temperature with thinking disabled is a common way to end up convinced a good model is bad.

Vision and context: what they actually cost you

Vision costs 0.86 GiB of weights plus whatever the image consumes in context; long context costs something we have not been able to pin down, so start conservative.

The vision half is easy to budget. The projector file, mmproj-F16.gguf, is 0.93 GB — 0.86 GiB. Load it alongside the weights and image understanding works; skip it and you have a text model. On a 24GB card with UD-Q4_K_XL that is 17.55 GiB of the 24 GiB spoken for, which is comfortable. On a 16GB card with UD-Q3_K_XL it is 13.38 GiB of 16, which is workable but tightens your context budget considerably.

Context is the honest gap in this page. Qwen3.8-27B is not a conventional all-attention transformer: the model card describes 16 repeats of a block containing three Gated DeltaNet layers for every one Gated Attention layer, so only 16 of the 64 layers carry a standard KV cache. That is exactly the design choice that makes a 262K native window plausible on a consumer card at all. But neither Qwen nor Unsloth publishes a KV-cache-per-token figure for this architecture, and we have not measured it ourselves yet, so we are not going to invent a number for “VRAM at 256K context”.

Practical advice instead: start at 32K, watch actual VRAM usage in nvidia-smi or Activity Monitor, and raise it until you are about 1GB from the ceiling. If you need the full 262K, or the ~1M that YaRN extension unlocks, you want a 32GB card or a large unified-memory box rather than a 24GB one — see the hardware hub for what that costs in the current GPU pricing situation.

Qwen3.6-27B → Qwen3.8-27B: worth swapping?

Yes, if you are already on a 24GB card — it is a drop-in replacement at the same file size that adds vision and doubles the context. Qwen3.6-27B Q4_K_M was around 17GB; Qwen3.8-27B Q4_K_M is 17.11 GB. Same tier, same card, same install ritual.

Every number above is from the comparison table Qwen publishes on the Qwen3.8-27B model card, run by Qwen on their own harnesses. We have not independently reproduced any of them. Vendors pick benchmarks that flatter their release; the direction is almost certainly right, the margins may not be.

The concrete, non-benchmark upgrades are easier to trust: native image and video input where 3.6 had none, 262,144 tokens of native context against 3.6’s 128K, and reasoning_effort as a real dial rather than an on/off switch. Our existing Qwen3.6-27B page stays accurate for anyone who wants to stick, and the Qwen local setup guide covers the family-wide plumbing.

Where it is worse, or just annoying

• •It thinks by default, at maximum effort. reasoning_effort defaults to xhigh. Out of the box, a one-line question can produce a long reasoning trace before the answer. Set it to low or medium for anything conversational — in llama.cpp that is --chat-template-kwargs ‘{"reasoning_effort":"medium"}’.
• •Preserved thinking eats context. The model retains reasoning blocks from every prior turn by default. Qwen argues this improves consistency in agent loops, and it does help KV cache reuse — but in a long chat it quietly consumes your window. Disable with preserve_thinking: false if you notice sessions getting expensive.
• •No small sibling. Covered above, but it is the single biggest practical drawback of this release for anyone below 16GB.
• •It does not top every chart. On Qwen’s own table, Opus 4.6 Max leads on Terminal Bench 2.1, HLE, GPQA Diamond and repo-level generation; Qwen3.7-Plus leads on OmniDocBench and RealWorldQA. This is an excellent model for its memory footprint, not a frontier-beater.
• •Consumer speed is unmeasured. The only throughput figures we could find in Unsloth’s Qwen3.8 documentation are for the 2.4T MoE on B200s, not for the 27B — and a datacenter GPU tells you nothing reliable about your 4090 anyway. Nobody, us included, has published tokens-per-second for this model on consumer cards yet, so we are not going to invent one. We will add measured numbers when we have run them ourselves.

Frequently asked questions