Meta's Muse-Glimmer 30B on a 16GB GPU: Which Quant Actually Fits

The full quant ladder, file by file

There are fifteen text builds in the unsloth repo spanning 10.75 GB to a 55.73 GB BF16 split, and only two in Meta's own. That is the whole story of this release for anyone on consumer hardware: Meta optimised for the 24/32GB envelope and left everything below it to the community. Here is the ladder as it exists today.

The two companion files nobody budgets for

Meta publishes its own combined figures for the two official builds: the 17GB build runs about 17 GB text-only, 19 GB with vision, 20 GB with vision and drafter; the dynamic build 20 / 22 / 23 GB. Those are the numbers to plan a 24GB or 32GB card around. For a wider view of what different cards can hold, the full Ollama RAM and VRAM table is the reference we keep updated.

Specs

29.6B total parameters including a ~1.8B ViT-G/14 perception encoder, 131,072+ context, Apache 2.0, knowledge cutoff 4 January 2026. Everything below is from the model card.

The attention pattern is the underrated line in that table. Three of every four layers use a 2,048-token sliding window, and the global layers use GQA with just 2 KV heads. That combination is why Meta's own README can casually suggest running four concurrent 131K slots and describe the cost as "a few GB, not tens." Long context on this model is genuinely cheap. Since it takes images too, it belongs in the same conversation as the other local vision-language models, not just the text-only lineup.

Setup on a 16GB card

Check your llama.cpp build number first — anything before b10353 will not load these files. Meta's README says architecture support was merged on 10 August 2026 and first shipped in b10353. Builds b10344 and older do not register muse-glimmer at all.

./llama-cli --version          # must be >= b10353
# building from source instead? confirm the checkout:
grep -c LLM_ARCH_MUSE_GLIMMER src/llama-arch.cpp   # expect >= 1

Download the 16GB-friendly build

pip install huggingface_hub

hf download unsloth/Muse-Glimmer-30B-GGUF \
    --local-dir Muse-Glimmer-30B-GGUF \
    --include "Muse-Glimmer-30B-UD-Q3_K_XL.gguf"

Add --include "mmproj-kquant.gguf" only if you need image input, and expect to step down to UD-IQ2_M to pay for it.

Serve it

./build/bin/llama-server \
    -m Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q3_K_XL.gguf \
    -a muse-glimmer-30B \
    -ngl 99 -c 32768 -np 1 \
    --host 127.0.0.1 --port 8080 \
    --jinja \
    --temp 1.0 --top-p 0.95 --top-k 64

Three things in that command are not optional and all three are documented traps in Meta's README:

• 1.--jinja. The chat template is embedded in the GGUF. Without this flag, llama-mtmd-cli aborts outright and llama-server output starts with raw to=self<|message|> markers instead of clean text.
• 2.-c is divided by -np. A single request gets -c / -np, and the startup log's n_ctx_slot is the number that actually bounds a generation. Get this wrong and long requests silently return nothing — no error, just an empty answer.
• 3.Never add <|eom|> as a stop string. The stop tokens are <|end_of_text|> (200001) and <|eot|> (200008). <|eom|> is end-of-message, not end-of-turn — Meta's README says the turn continues after it, so stopping there truncates the response mid-turn.

Turning the thinking down (you cannot turn it off)

# server-wide
--chat-template-kwargs '{"reasoning_strength":"low"}'

# or per request
{"chat_template_kwargs": {"reasoning_strength": "medium"}}

# hard ceiling on thinking tokens
--reasoning-budget 2048

Levels are low / medium / high / xhigh, defaulting to high. On a 16GB card with a 32K window, dropping to low or medium for routine work is the difference between an answer and an empty response.

Images need a different binary

./build/bin/llama-mtmd-cli \
    -m       Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-IQ2_M.gguf \
    --mmproj Muse-Glimmer-30B-GGUF/mmproj-kquant.gguf \
    -ngl 99 -c 16384 --jinja \
    --temp 1.0 --top-p 0.95 --top-k 64 \
    --image screenshot.png -p "What is this dialog asking me to do?"

Apple Silicon and the Ollama route

Ollama has Muse-Glimmer with 15 tags and roughly 126K pulls, but its smallest is 17 GB — so Ollama is a 24GB+ or 32GB-unified-memory path, not a 16GB one. If you have the memory, it is by far the least painful install.

ollama pull muse-glimmer          # 18GB, 128K context, text + image
ollama run muse-glimmer

On Macs, the practical floor is 32GB of unified memory for the 18GB default. A 24GB Mac will technically load it and then fight the OS for the remainder. Meta's own speed measurements were taken on M4 Max and M5 Max machines using ExecuTorch — if you are working out which Apple chip to buy for this kind of workload, our Apple M5 for local AI guide covers the memory-bandwidth side.

Speed and the DFlash drafter

Meta reports 74.9 tok/s on an RTX 5090 without speculation and 233.4 tok/s with the DFlash drafter — a 3.1x speedup. These are vendor figures, measured on the K-Quant-17GB build at batch size 1 with greedy decoding. We have not reproduced them.

The drafter is a small block-diffusion network based on Meta's DFlash work: 5 layers, tapping hidden features from layers 1, 13, 25, 37 and 49 of the 52-layer parent, predicting a 16-token block per forward pass which the big model then verifies in parallel. Output is supposed to be identical to unspeculated decoding — you are buying throughput, not a different model.

Wiring it up in llama.cpp is two extra flags:

hf download meta-models/Muse-Glimmer-30B-GGUF \
    --local-dir Muse-Glimmer-30B-GGUF \
    --include "dflash-Muse-Glimmer-30B-Q4_K_M.gguf"

# then append to your llama-server command:
    -md Muse-Glimmer-30B-GGUF/dflash-Muse-Glimmer-30B-Q4_K_M.gguf -ngld 99

A [spec] failed to measure draft model memory warning at startup is expected and harmless per Meta's README. One correction worth making: some coverage describes the companion as a "3B assistant model" you can chat with. It is published as meta-models/Muse-Glimmer-30B-assistant, but its own card and config make clear it is the DFlash drafter head — 5 layers, block size 16 — not a standalone small model. Do not plan to run it on its own.

Benchmarks (all vendor-reported)

Muse-Glimmer sweeps the agentic column and loses the headline coding and computer-use benchmarks to Qwen3.6-27B. Every number below is Meta's, from the model card, with Muse-Glimmer at high reasoning strength and the comparators in thinking mode. Treat them as a vendor's framing of its own release.

Read that table as a shape, not a scoreboard. Muse-Glimmer is tuned for multi-step tool loops and it shows: +13 points on MCP Atlas over Qwen3.6-27B is a big gap. But if your actual job is "fix this failing test," Qwen3.6-27B is still ahead on SWE-Bench Verified and nine points ahead on TerminalBench. Our Qwen3.6-27B page has the other side of that comparison, and our best-models roundup places both against the rest of the field.

Apache 2.0 from Meta — why that is the actual story

Every released artifact is Apache 2.0: BF16 weights, both 4-bit builds, the DFlash drafter head and the perception encoder. The model card says so in its Released Artifacts table, and the HF repo carries the apache-2.0 licence tag.

If you have been shipping on Llama weights, you know why that matters. The Llama community licence is a bespoke agreement with conditions attached — the sort of thing that makes a legal review a project rather than a checkbox. Apache 2.0 is a licence your company's counsel has already approved for a hundred other dependencies. It grants commercial use, modification, redistribution and a patent grant, and the only real obligations are attribution and preserving the notice.

Two caveats so you do not over-read it. Meta ships a separate USAGE_POLICY.md in the repo and the card's out-of-scope section directs you to it — that is a policy document alongside the licence, not inside it, so read it before building a product. And the card states the model is not intended to be downloaded or used by anyone under 18, and places responsibility on deployers whose systems might reach minors. Neither of those changes the licence on the weights, but both belong in your risk write-up.

Honest limitations

• •16GB is a compromise tier, not a supported one. Meta's degradation table only measures its own two 4-bit builds (1.0% and 0.2% average degradation across 15 benchmarks). Nobody has published equivalent numbers for IQ2 or Q3 on this model. A 13.4GB Q3 build of a 29.6B model will be measurably worse than the 16.8GB one, and how much worse on your workload is something you have to check yourself.
• •Vision is not free. The perception encoder is a separate 1.40GB file that has to sit in memory alongside the weights, and each image can consume up to 4,096 visual tokens of your context. On a 16GB card that pushes you down a quant rung and eats the window.
• •Thinking cannot be disabled. Only turned down. If you wanted a fast non-reasoning model for classification or extraction, this is the wrong one.
• •Running out of context fails silently. Meta warns about this directly: when a generation exhausts its slot, the request produces no answer and nothing appears in the logs. In an eval harness that reads as a wrong answer rather than a failure.
• •No audio, and video only as frames. The card states audio in/out is out of scope and that video is processed as individual frames with no explicit video optimisation.
• •Knowledge cutoff is 4 January 2026. For agent work that mostly does not matter — it is calling tools, not reciting facts — but do not expect it to know anything about 2026 releases, including its own.

Frequently asked questions