Best Local Model for a 24GB GPU Right Now: Qwen3.8-27B vs Muse-Glimmer 30B vs Nemotron 3.5

Short answer: on a 24GB card, pull Qwen3.8-27B. Its Unsloth UD-Q4_K_XL GGUF is 17.92GB, the vision projector adds 0.93GB, and Ollama's one-command pull is 18GB at a 256K context window — the only one of the three that leaves comfortable KV-cache headroom while still doing images. Muse-Glimmer 30B (UD-Q4_K_XL, 15.88GB) is the better pick for long agent runs. Nemotron 3.5 Lightning fits 24GB only via the official ggml-org Q4_0 at 18.90GB — its default Ollama pull is 25GB and will not fit.

Three 30B-class open-weight models landed on Hugging Face inside two weeks, and all three were designed for the same slot: one consumer GPU with 24GB. Alibaba's Qwen repo went up 5 August. Meta Superintelligence Lab's Muse-Glimmer followed on 9 August. NVIDIA dated Nemotron 3.5 Lightning's release 11 August. GGUF conversions arrived between 10 and 13 August, which is when this comparison actually became possible.

This page is the decision, not the setup depth. It is decided on file sizes pulled from the live GGUF repositories, because on a fixed-VRAM card what fits beats what benchmarks.

---

The 30-Second Verdict

Pick by what you are doing, not by the leaderboard — the three barely overlap in purpose.

---

What Fits, By VRAM Tier

Every size below is a real file size read from the live GGUF repositories on 18 August 2026, not an estimate. Weights are only part of the story — you still need KV cache on top, and for the two vision models you need the separate mmproj projector file if you want image input at all.

Sources: unsloth/Qwen3.8-27B-GGUF, unsloth/Muse-Glimmer-30B-GGUF, ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF and bartowski/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF.

Reading the 24GB row properly. Qwen3.8 at 17.92GB plus the 0.93GB projector is ~18.9GB of weights, leaving roughly 5GB for context — that is a genuinely comfortable 24GB fit and why it is the default recommendation. Muse-Glimmer is the roomiest at 15.88GB, but if you want both image input and the DFlash drafter you add 1.40GB and 1.63GB and end up at ~18.9GB too. Nemotron's 18.90GB is text-only weights with nothing optional to drop, so it is the tightest of the three despite being nominally the same tier.

The 12GB row is a warning, not a recommendation. Two-bit quantisation of a 27-30B model is a real quality cliff, and neither vendor validated it. If you have a 12GB card, our RTX 5070 write-up and the 24GB VRAM ladder will point you at models that were actually designed for the space you have.

---

The Nemotron Quant Trap

Nemotron 3.5 Lightning has effectively one size, around 19GB, and no smaller quant will save you. This is the single most surprising thing we found, and it is easy to verify.

Look at the bartowski repository file listing:

An IQ2_XXS that is only 0.2GB smaller than a Q4_0 is not a rounding error — it is the shape of the model. Unsloth's repo shows the same floor from the other side: its UD-IQ1_M, UD-IQ2_M and UD-IQ2_XXS files are all exactly 19.43GB, and its UD-Q4_K_M is 25.27GB.

Why the floor exists is our reading rather than a published explanation. NVIDIA's card does state that "the pre-training phase used an NVFP4 recipe" on a Mamba-2 + MoE hybrid, and a model whose bulk tensors were already trained near 4-bit has very little left for a 2-bit quantiser to take. NVIDIA does not spell that out as the cause, and we have not measured it — but whatever the mechanism, the file listings are the file listings, and they say Nemotron has one size.

Two practical consequences:

1. Do not pull nemotron-3.5-lightning:latest on a 24GB card. Ollama's default tag is a 25GB Q4_K_M. Use the official ggml-org Q4_0 through llama.cpp instead, or accept the CPU offload.
2. There is no 16GB Nemotron. Not at IQ2, not at IQ1. If your card is 16GB, this model is off the table unless you are offloading to system RAM — which, to be fair, it tolerates better than the other two.

---

Setup: All Three

Every one of these is a single command if you use Ollama, and all three tags were verified live on ollama.com.

# Qwen3.8-27B — 18GB pull, 256K context, text + image input
ollama run qwen3.8

# Muse-Glimmer 30B — 18GB pull, 128K context, text + image input
ollama run muse-glimmer

# Nemotron 3.5 Lightning — 25GB default pull, 1M context, text only
# WARNING: this default does NOT fit a 24GB card
ollama run nemotron-3.5-lightning

For Nemotron on a 24GB card, use NVIDIA's own llama.cpp recipe against the official GGUF instead. This is the "Local AI (RTX 5090, DGX Spark, and RTX 6000 Pro)" section of NVIDIA's NVFP4 model card — not the BF16 card, which carries no llama.cpp instructions at all — with one correction of ours:

llama-server \
  -hf ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF:Q4_0 \
  --temp 1.0 --top-p 0.95 \
  -np 1 -c 40960 --port 8000 \
  -ngl 99 -fa on --jinja --no-webui --fit off

The correction matters on a 24GB card. NVIDIA's card writes that line as ...GGUF:Q4_K_M, but as of 18 August 2026 the ggml-org repository holds only three weight files — Q4_0 (18.90GB), Q8_0 (33.59GB) and BF16 (63.18GB). There is no Q4_K_M in it. Pin :Q4_0 and you get the file that fits; leave the tag off and llama.cpp picks for you.

Sampling settings matter more than usual here, because all three ship with thinking enabled by default:

• Qwen3.8 thinking mode: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0. Non-thinking: temperature=0.7, top_p=0.80, top_k=20, presence_penalty=1.5. It also exposes a reasoning_effort parameter with three levels — xhigh (the default), medium and low.
• Muse-Glimmer: temperature=1.0, top_p=0.95, top_k=64. Reasoning strength is set in the system prompt as Reasoning strength: <value> with levels low / medium / high / xhigh — an unusual convention worth knowing about before you file a bug.
• Nemotron: temperature=1.0, top_p=0.95, with thinking toggled via enable_thinking in the chat template.

One Ollama-specific gotcha, again from NVIDIA's NVFP4 card rather than the BF16 one: Ollama sets Nemotron's context dynamically from available VRAM — under 24GB it gives you 4K, 24-48GB gives 32K, 48GB+ gives 256K, and you can override it with /set parameter num_ctx at the cost of CPU offload. If your "1M context model" is truncating at 4K, that is why. New to this stack? Start with our complete Ollama guide.

---

What Each One Is Actually For

These three are not competitors so much as three different bets on what a 30B model is for.

Qwen3.8-27B — the generalist

A dense 27B with a native vision encoder, 64 layers, and an unusual hybrid layout the card describes as repeating blocks of Gated DeltaNet and Gated Attention. It takes images and video as input, natively handles 262,144 tokens of context and is described as extensible to 1M, and it is the only one of the three whose card talks about hour-scale video understanding. If you want one model on the box that does documents, screenshots, code and long context, this is it. It supersedes the model on our Qwen3.6-27B page, which is the direct predecessor.

Muse-Glimmer 30B — the agent

~29.6B parameters including a ~1.8B ViT-G/14 perception encoder, 52 layers, sliding-window attention at 2048 with every fourth layer global, 131,072+ context, knowledge cutoff 4 January 2026. Meta's framing is unusually specific: it is distilled from Muse Spark and trained for failure recovery — diagnosing a failed tool call and retrying rather than halting. It ships with a DFlash block-diffusion drafter for speculative decoding that predicts 16 tokens per forward pass.

The most useful thing on its card is not a benchmark, it is a quantisation table. Meta states K-Quant-Dynamic targets 32GB VRAM with 0.2% average degradation, and K-Quant-17GB targets 24GB with 1.0% degradation, measured across 15 benchmarks. A vendor publishing its own quality-loss-per-quant numbers is rare and worth more than another leaderboard row.

Nemotron 3.5 Lightning — the efficiency play

30B total, 3B active, hybrid Mamba-2 + MoE + attention, MTP layers, up to 1M context, and no vision at all. Six languages (English, Spanish, French, German, Italian, Japanese) versus 100+ for Muse-Glimmer. On paper it is the weakest of the three, and NVIDIA's own comparison table has it behind Qwen 3.6 35B A3B on most rows.

Its case is arithmetic, not accuracy: with 3B active parameters, the fraction of the model that has to move per token is tiny, so it degrades far more gracefully than a dense 27B when weights spill into system RAM. On a 16GB card with offload, or on a CPU-heavy box, that is the difference between usable and unusable. See running LLMs on CPU only for what that path really looks like.

---

Benchmarks, With Caveats

Read these as vendor claims, because that is exactly what they are — nobody re-ran anybody else's harness.

Qwen's card is the only one that benchmarks a rival directly. These are Qwen's numbers, from Qwen's harness — a subset of its table, which also carries Qwen3.7-Plus and Opus 4.6 Max columns we have dropped because neither is a local model:

Meta's card runs a different comparison — against Gemma4-31B and Qwen3.6-27B, not 3.8 — and unsurprisingly Muse-Glimmer leads on the agentic rows it was built for: MCP Atlas 75.5 (vs 62.5 for Qwen3.6), DeepSearch QA 74.6, AA-LCR 80.0, AIME 2026 94.7. On OSWorld-Verified and TerminalBench 2.1 it puts Qwen3.6 ahead of Muse-Glimmer.

That last point is the cleanest illustration of why vendor tables are not comparable. Both cards report Muse-Glimmer's OSWorld-Verified score as 65.9 — they agree exactly. They disagree about the rival: Qwen scores Qwen3.6-27B at 63.9 on OSWorld, Meta scores the same model at 75.6. Same benchmark, same two models, an 11.7-point gap in the number each vendor got for the other side. Nobody is lying; the harnesses differ. That is the size of the noise you are reading through on every row here.

NVIDIA's card is the most candid of the three. It states the accuracy numbers were "measured by NVIDIA under a consistent harness (NeMo Gym / Nemo Evaluator SDK)" and that "they may differ from vendors' self-reported numbers", then publishes results that put Nemotron behind Qwen 3.6 35B A3B on MMLU Pro (81.94 vs 85.63), GPQA Diamond (75.44 vs 83.40), SWE-bench Verified (51.56 vs 70.12) and Terminal-Bench 2.1 (24.58 vs 44.38). Publishing a table you lose is a decent signal about the rest of the table.

Our take: the gaps big enough to trust are Qwen3.8 being clearly ahead on document and chart understanding, and Nemotron being clearly behind on agentic coding. Everything inside a few points is harness noise. For the wider field see our best AI models roundup.

---

Licences

Two Apache 2.0, one not.

OpenMDW is a model-and-data licence rather than a plain software licence, and "commercial use is permitted" is not the same claim as "this is Apache". If Nemotron is going into a product, read the actual text — the runtime you serve it with being permissive has no bearing on what the weights allow.

---

Apple Silicon

On a Mac you need 32GB of unified memory to be comfortable, and 24GB is the edge. Ollama publishes MLX builds for all three, and the sizes are close to the CUDA ones:

Unified memory is shared with the OS and everything else you have open, so a 24GB Mac running an 18GB model is tight in a way a 24GB discrete card is not. Meta's own figures on the same hardware class are sobering next to the CUDA ones: 23.7 tok/s baseline on an M4 Max and 26.6 on an M5 Max, versus 74.9 on an RTX 5090 — all vendor-measured, Macs via ExecuTorch and the 5090 via llama.cpp. Our Apple Silicon buying guide covers the memory-tier decision properly.

---

Honest Limitations

The things nobody puts on a model card.

• All the speed numbers on this page are vendor-measured. We have not run our own tok/s harness against these three. Where we quote 233.4 tok/s on a 5090, that is Meta's figure, batch size 1, greedy decoding, on Meta's prompt set, with a separate drafter loaded.
• Vision costs more than the projector file. Image tokens consume context, and Muse-Glimmer's card caps a single image at 4,096 visual tokens. On a 128K-context model with a 24GB budget, a handful of screenshots is not free.
• Thinking-by-default eats your context and your latency. All three reason before answering unless you turn it off. Qwen's own card warns that lower reasoning effort does not always reduce total task time in multi-turn agent work, because weaker analysis causes retries.
• These models are days old. Quant repos are still being updated — the Unsloth Qwen3.8 repo was last modified on 15 August, days after it was created. Re-check file sizes before you plan a build around a specific number.
• Nemotron has no vision and six languages. If either of those is a requirement, the comparison is over before it starts.
• We could not verify third-party benchmarks for any of the three. At the time of writing there are no independent evaluations to check the vendor numbers against.

---

Sources

• Qwen/Qwen3.8-27B — architecture, context length, licence, benchmark tables, sampling settings
• meta-models/Muse-Glimmer-30B — architecture, quantisation/degradation table, DFlash speedups, benchmarks
• nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 — release date (11 August 2026), MoE details, NVFP4 pre-training recipe, OpenMDW-1.1 licence, NeMo Gym benchmarks
• nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 — the "Local AI" section: the llama.cpp server recipe and the Ollama context-by-VRAM tiers
• GGUF file listings: unsloth/Qwen3.8-27B-GGUF, unsloth/Muse-Glimmer-30B-GGUF, ggml-org/ and bartowski/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF
• ollama.com/library — tag names, pull sizes and context windows for all three

---

FAQ