Meta says 3.1x but doesn’t say of what: running Muse Glimmer 30B locally with llama.cpp
Meta Superintelligence Labs released Muse Glimmer on August 10: a dense 30-billion-parameter model, Apache-2.0, distilled by logits from Muse Spark, multimodal and trained on data from over 100 languages. The framing in the blog itself is unusually narrow for a model release — this isn’t presented as a general-purpose 30B, it’s presented for a single job: “Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows.”
The technical argument behind that pitch is a memory envelope. At full precision the model needs over 55 GB. Quantized to 4-bit with K-quants, the language model drops below 20 GB, which fits it inside 24 and 32 GB machines. And for an always-on agent to be tolerable at that size, Meta includes a speculative decoding drafter, DFlash, with three published speedups: 3.1x on an RTX 5090, 1.8x on an M5 Max, 1.5x on an M4 Max.
Those are the numbers from the announcement. There are two things worth knowing about them before planning a machine around them.
First: the speedups have no denominator. Meta publishes multipliers and no absolute throughput anywhere — neither in the blog nor in the model card. A 3.1x over a tok/s that isn’t declared tells you the drafter works; it doesn’t tell you whether the agent is usable on your machine. You have to measure that number yourself, and the last section of this article is how.
Second: the “under 20 GB” is only the text model. The vision encoder and the drafter are separate files, and each one is memory you also have to sustain. The real figure comes further down.
What actually came out, and what didn’t
The blog says that optimizations in llama.cpp, MLX and ExecuTorch “will land in the coming days”. That was true for about a day, and it’s already outdated just for the one that matters most.
llama.cpp support is merged and working today — build b10353 or later, landed August 10. Meta published official GGUFs at meta-models/Muse-Glimmer-30B-GGUF, and the community had conversions up the same day (bartowski, unsloth). MLX and ExecuTorch are still genuinely pending. So if you’re on Apple silicon and were expecting MLX, you’re still waiting; if you want to run this today on anything, llama.cpp is the way, Metal included.
That’s the difference between an announced integration and a merged one, and on this release it cuts both ways.
How much machine you actually need
Meta’s official GGUF repo brings four files. This is the table the “under 20 GB” headline compresses:
| File | Size | What it is |
|---|---|---|
muse-glimmer-30B-kquant-17gb.gguf |
16.8 GB | Text model, targeted at 24 GB VRAM |
muse-glimmer-30B-kquant-dynamic.gguf |
19.7 GB | Higher quality variant, targeted at 32 GB |
mmproj-kquant.gguf |
1.4 GB | Vision encoder — necessary for images |
dflash-kquant.gguf |
1.6 GB | DFlash speculative decoding drafter |
Which gives three real configurations, according to Meta’s own stated requirements:
- Text only — ~17–20 GB
- Text + vision — ~19–22 GB
- Text + vision + drafter — ~20–23 GB
And that’s before the KV cache. Notice what it means for the 24 GB card the 17 GB quant targets: with vision and drafter loaded you’re at ~20–23 GB of weights, and the context window you can afford is what’s left over. Muse Glimmer supports 131,072 tokens of context and up to 262,144 — you’re not going to serve the ceiling of that range on a 24 GB card with the vision encoder resident. The 32 GB envelope is where the full configuration stops being a negotiation.
This is a feasibility number, not a speed number. It tells you if the model fits, and it says nothing about how fast it runs once it does.
If you want more headroom, Unsloth’s dynamic quants go lower: UD-Q2_K_XL at 12–14 GB, UD-Q3_K_XL at 14–15 GB, UD-Q4_K_XL at 17 GB (their recommended starting point), UD-Q6_K_XL at 20–22 GB, UD-Q8_K_XL at 34 GB. A note from their documentation: NVFP4 is marked as work-in-progress and doesn’t work right now. Don’t plan around that.
Start with the 17 GB quant even if you have 32 GB. It downloads faster, leaves you room to add the vision encoder and drafter one at a time, and you’ll learn where your own ceiling is by walking toward it instead of starting above it.
How to run it
The minimal path is one command, once you have a current llama.cpp:
# build b10353 or later required
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON # Metal is enabled by default on macOS; remove the flag for CPU-only
cmake --build build --config Release -j
Then, the model:
llama serve -hf meta-models/Muse-Glimmer-30B-GGUF
That downloads the default quant and leaves you with an OpenAI-compatible endpoint. Meta’s recommended sampling settings are temperature 1.0, top-p 0.95, top-k 64 — it’s worth setting them explicitly, because running a 30B with defaults tuned for another model is a common way to conclude that a model is worse than it is.
To add vision, load the multimodal projector alongside the text weights:
llama-server \
-m muse-glimmer-30B-kquant-17gb.gguf \
--mmproj mmproj-kquant.gguf \
-c 32768
Keep the context moderate at first. -c 32768 on a 24 GB card is a realistic starting point with vision loaded; raise it until you hit a memory allocation failure and then drop back one notch. Keep in mind that video is processed as individual frames only — there’s no temporal modeling here.
To add the drafter, DFlash goes in as the draft model:
llama-server \
-m muse-glimmer-30B-kquant-17gb.gguf \
--model-draft dflash-kquant.gguf \
-c 32768
It’s worth understanding the mechanism, because it explains where the speedup comes from and when it evaporates. DFlash is a 5-layer drafter with sliding-window attention that predicts entire blocks of 16 tokens in a single forward pass; then the 30B verifies the block all at once. When the drafter guesses right, you get many tokens for each expensive forward pass. When it guesses wrong, you pay the drafter pass and get little back. Speculative decoding is a bet on predictability — which is exactly why it performs well on agentic workloads full of structured tool calls and JSON, and worse on open prose.
That also explains the difference between hardware: 3.1x on an RTX 5090 versus 1.8x on an M5 Max isn’t the drafter behaving differently, it’s how much the base decode step was memory-bandwidth-bound.
Measuring the number Meta didn’t publish
Since there’s no tok/s reference anywhere in the entire release, here’s how you get yours. llama-bench gives you prefill and decode separately:
llama-bench -m muse-glimmer-30B-kquant-17gb.gguf -p 512 -n 128
pp512 is prefill (prompt processing), tg128 is decode (token generation). They’re different numbers with different bottlenecks and you want both — an agent that reads a large context before each action lives or dies by prefill, not decode.
Then run the same thing with the drafter connected and compare. Your multiplier won’t be 3.1x unless you’re on an RTX 5090 running a workload as predictable as Meta’s, and that’s exactly the point: the multiplier is a property of your workload, not the model.
Do the measurement with the actual prompts your agent sends. A tool-calling loop and a chat session are going to give you quite different acceptance rates.
Is it actually better than what you’re already running?
Meta’s benchmark table compares Muse Glimmer against Gemma4-31B and Qwen3.6-27B. All figures are self-reported by Meta:
| Benchmark | Muse Glimmer | Gemma4-31B | Qwen3.6-27B | |
|---|---|---|---|---|
| Agentic | MCP Atlas | 75.5 | 54.2 | 62.5 |
| DeepSearch QA | 74.6 | 61.7 | 71.1 | |
| Coding | SWE-Bench Pro | 51.2 | 36.9 | 50.2 |
| SWE-Bench Verified | 76.0 | 66.6 | 77.2 | |
| Multimodal | Charxiv Reasoning | 78.8 | 77.7 | 78.4 |
| MMMU Pro | 74 | 73 | 75 | |
| Reasoning | AIME 2026 | 94.7 | 89.2 | 94.1 |
| GPQA Diamond | 83.5 | 85.7 | 84.2 |
Read that table honestly and say something more useful than “Meta wins”. In coding, multimodal, and reasoning, Muse Glimmer and Qwen3.6-27B trade positions within a point or two, and Qwen takes SWE-Bench Verified and MMMU Pro. Gemma4-31B takes GPQA Diamond. Those columns describe three models in the same class.
The only place where the gap isn’t marginal is agentic: MCP Atlas 75.5 versus 62.5 for Qwen — thirteen points — and DeepSearch QA at 74.6 versus 71.1. That’s where mid-training with agent-heavy long-context data and post-training RL show up, and it’s the only column that justifies the framing of the launch.
The HN thread (1,123 points, 609 comments) landed on the same reading from the other side: users describe Glimmer barely above Qwen3.6-27B on almost everything except tool-calling. No Meta authors appear in the thread. Other reports worth weighing: quantized sizes land close (a Q4_K_XL at ~15.9 GB versus Qwen3.6-27B’s 17.6 GB), KV cache limits practical parallel inference to about 4 concurrent requests on consumer hardware, and Glimmer’s reasoning traces are consistently described as terse.
So the decision is narrower than the announcement suggests. If you run a local model for chat, code completion, or writing, the case for switching is thin — a couple of benchmark points, either way. If you run a local model as an agent — MCP servers, tool loops, multi-step workflows — that thirteen-point gap on MCP Atlas is the whole reason this model exists, and it’s worth measuring against your own setup.
What to check before you commit
Three things this launch doesn’t tell you, in order of how much they’ll cost you:
- Absolute throughput. Measure it. A 3.1x on a machine decoding at 4 tok/s is a different product than a 3.1x on one decoding at 40.
- Your prefill. Agent loops reread context all the time. Nobody published a prefill number and
llama-bench -pgives you yours in a minute. - Whether you need the vision encoder or not. It’s 1.4 GB of resident memory plus its share of overhead. If your agent never sees an image, leaving
mmprojunloaded gives you context back.
And if you’re on Apple silicon: llama.cpp with Metal works today, MLX doesn’t yet. Worth knowing which you’re waiting for before you plan around a number.
If you already run a local model as an agent — how much of your setup is bottlenecked by tokens per second, and how much by the model just picking the wrong tool? Tell us what you’re running and what for.