Muse Glimmer 30B on RTX 5090: K-Quant-Dynamic, vision and DFlash in llama.cpp

What You'll Build

An OpenAI-compatible llama-server on a single RTX 5090 running Meta's Muse Glimmer 30B — an agentic model that takes interleaved text and images and emits tool calls — loaded from Meta's own K-Quant-Dynamic GGUF, with the vision projector attached and the DFlash block-diffusion drafter wired in for speculative decoding.

This is deliberately the 32 GB configuration. Meta ships two 4-bit builds and targets them at different cards: K-Quant-17GB at 24 GB and K-Quant-Dynamic at 32 GB, with the quantisation section of the model card noting "This leaves enough headroom for the model's KV cache, the perception encoder for image understanding, and the speculative decoding drafter to run simultaneously within a 24 GB or 32 GB envelope." The RTX 5090 is the consumer card that gets the larger, less lossy one — the card puts K-Quant-Dynamic at 0.2% average degradation across its 15-benchmark suite against 1.0% for K-Quant-17GB.

Hardware data: RTX 5090 (32 GB VRAM) · derived working set 22.90 GiB at the full 131,072-token context · See benchmark data

⚠️ Two known issues, both in Troubleshooting. Builds from before the architecture landed reject the file outright with llama_model_load: error loading model: unknown model architecture: 'muse-glimmer'. Separately, llama.cpp issue #26894 — open, and reported against this recipe's exact target and drafter files — has the DFlash drafter failing to bind on some builds. It does not reproduce universally, including for the author of the architecture's own support PR, and one reader on this page runs the configuration successfully. Read that entry before you commit to the speculative-decoding path.

ℹ️ Meta's headline speed numbers are for the other quant. The model card's RTX 5090 row (74.9 tok/s baseline → 233.4 tok/s with DFlash, a 3.1× speedup) is prefaced by "We measure the speed of our K-Quant-17GB model alongside the quantized DFlash drafter" — so it describes the 24 GB build, not the K-Quant-Dynamic weights this recipe installs. That table is the card's only speed table, so the card publishes no first-party tok/s figure for K-Quant-Dynamic on any hardware. It is at least the right runtime for this page: the card's own footnote reads "M4/M5 measurements were done using ExecuTorch, and RTX using llama.cpp." Treat the 233.4 as context, not as a target for this configuration, and read the Results section before you calibrate expectations.

Requirements

Where the 32 GB floor comes from

Every byte below is either a file size read from the HuggingFace tree API for meta-models/Muse-Glimmer-30B-GGUF, or arithmetic over the published architecture. Nothing here was measured on a card.

That leaves 9.10 GiB of the card's 32 GiB for the CUDA context, compute buffers and image tensors — comfortable, and the reason this quant belongs on a 5090 rather than a 24 GB card, where the same three files plus KV would land inside 1.1 GiB of the ceiling.

The KV figure is small for a 30B because of two architecture choices, both in the model's config.json: GQA at 32 query heads to 2 KV heads, and a repeating [local, local, local, global] layer pattern with a 2,048-token sliding window. Of 52 layers, 13 are full attention and 39 are sliding. llama.cpp models this with two separate caches (llama_kv_cache_iswa): the full-attention cache gets n_ctx cells, the sliding one gets GGML_PAD(min(n_ctx, n_swa + n_ubatch), 256) = 2,560 cells at the default -ub 512. At f16 one cell of one layer costs 2 × 2 KV heads × 128 head-dim × 2 bytes = 1,024 bytes, so:

• full-attention layers: 13 × 131,072 × 1,024 = 1,744,830,464 bytes
• sliding layers: 39 × 2,560 × 1,024 = 102,236,160 bytes

The drafter is 5 layers, all sliding, at 8 KV heads × 128 head-dim → 4,096 bytes per cell per layer, so 5 × 2,560 × 4,096 = 52,428,800 bytes. Going from 131,072 down to 32,768 context only saves 1.22 GiB, because 39 of the 52 layers do not grow with context at all — there is little reason to run this model short.

Installation

1. Build llama.cpp with CUDA for Blackwell

Support for this architecture merged as PR #26841 on 2026-08-10, in commit 62bf73d. That commit is build number 10349, but llama.cpp does not publish a tag for every build — b10345 through b10352 do not exist as tags — so the earliest release you can download with the architecture in it is b10353.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j
./build/bin/llama-server --version

2. Confirm your checkout actually has the architecture

Cheaper than discovering it at load time, and it answers the question for whatever build you happen to have:

grep -n 'LLM_ARCH_MUSE_GLIMMER' src/llama-arch.cpp
grep -n 'PROJECTOR_TYPE_MUSE_GLIMMER' tools/mtmd/clip-impl.h
grep -n 'COMMON_SPECULATIVE_TYPE_DRAFT_DFLASH' common/speculative.cpp

Three hits means the text model, the vision projector and the DFlash speculator are all present in your tree. On master at 9558fa44 all three are there, with a dedicated clip_graph_muse_glimmer builder and a mtmd_image_preprocessor_muse_glimmer — vision for this model is implemented in mainline mtmd, not merely permitted by the presence of an mmproj file.

3. Download the three first-party GGUF files

hf download meta-models/Muse-Glimmer-30B-GGUF \
  Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf \
  mmproj-Muse-Glimmer-30B-Q4_K_M.gguf \
  dflash-Muse-Glimmer-30B-Q4_K_M.gguf \
  --local-dir ~/models/muse-glimmer-30b

If you reach for llama.cpp's -hf shorthand instead of downloading by name, tag it. A bare -hf meta-models/Muse-Glimmer-30B-GGUF looks for Q4_K_M and then Q8_0, and since the rename the only file in the repo matching either is the 24 GB-target Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf — so you would quietly get the smaller build this recipe is not about. -hf meta-models/Muse-Glimmer-30B-GGUF:Q4_K_XL picks the Dynamic build instead.

All three live in the same first-party repo under Apache 2.0. Use these names, not the *-kquant*.gguf ones. Meta republished the text and drafter builds under canonical Q4_K names on 12 August with a corrected embedded chat template; the superseded objects are still in the repo, and the GGUF card says of the old template that it "skipped both, which could leave a conflicting Reasoning strength: high. in the prompt" and to re-download if you pulled before the fix. The projector is the same object under both names — byte-identical, oid f48b452316f9… — so only the text and drafter files actually changed. The drafter is a genuinely separate model — its unquantised source is meta-models/Muse-Glimmer-30B-assistant, 5 layers with block_size: 16 and target_layer_ids: [1, 13, 25, 37, 49], and llama.cpp gives it its own GGUF architecture (dflash) rather than treating it as an ordinary small draft model.

Running

~/llama.cpp/build/bin/llama-server \
  -m  ~/models/muse-glimmer-30b/Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf \
  -mm ~/models/muse-glimmer-30b/mmproj-Muse-Glimmer-30B-Q4_K_M.gguf \
  -md ~/models/muse-glimmer-30b/dflash-Muse-Glimmer-30B-Q4_K_M.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 15 \
  --spec-draft-n-min 2 \
  -ngl all -ngld all \
  -c 131072 -np 1 \
  -fa on --jinja --reasoning-preserve \
  --host 127.0.0.1 --port 8080

The four flags that are not boilerplate:

• -md + --spec-type draft-dflash — both are required. -md alone is not enough: llama.cpp's speculative type defaults to none, and without the explicit --spec-type the drafter is either ignored or run as an ordinary autoregressive draft model. The reporter in discussion #34 describes exactly this — his server was silently defaulting to draft-simple until he added the flag, which he calls the key missing one. You can confirm it engaged from the startup log line common_speculative_impl_draft_dflash: adding speculative implementation 'draft-dflash'. If instead this is the step that crashes, jump to the drafter-binding entry in Troubleshooting — there is an open issue against exactly this pairing.
• --spec-draft-n-max 15 — DFlash denoises a block of block_size tokens in place and yields at most block_size - 1 of them, so with block_size: 16 the ceiling is 15. Ask for 16 and llama.cpp clamps it and logs a warning (common/speculative.cpp).
• -mm — the vision projector. Drop it and you have a text-only server; the model's own tokenizer still carries image and video token ids either way, so nothing will complain.
• -np 1 — a single slot keeps the whole KV budget on one sequence, which is what the derivation above assumes.

Sampling follows the model card's recommendation: --temp 1.0 --top-p 0.95 --top-k 64. Set reasoning strength through the template variable rather than by hand-writing a line into your system prompt — --chat-template-kwargs '{"reasoning_strength":"xhigh"}' server-wide, or "chat_template_kwargs": {"reasoning_strength": "high"} per request. It defaults to high, and xhigh suits coding and agentic work. Reasoning cannot be turned off at all; to cap it, use --reasoning-budget N. --jinja is what activates the embedded template, so it is not optional here.

For images, POST to /v1/chat/completions with an image_url content part in the usual OpenAI shape — llama-server is one of the binaries llama.cpp documents as multimodal-capable. Beyond the source-level support, one first-hand report of the vision path working end to end with this mmproj comes from a reader running the K-Quant-17GB build on an RTX 3090 in discussion #13; that is a different card, so take it as evidence that the path works, not as a number for this one.

Results

Speed — three independent RTX 5090 reports, all community, all on llama.cpp. The backend has no benchmark row for this pair yet (/check/muse-glimmer-30b/rtx-5090 returns verdict: unknown with an empty benchmark list), so everything below is attributed to the person who ran it:

Two things to take from that table. First, the baselines reproduce Meta's 74.9 tok/s closely, and the second reporter says so himself: "consistent with Meta's published 74.9 tok/s baseline". Second, the DFlash speedup is prompt-dependent to a degree the single headline number hides. A Meta engineer replying in discussion #21 confirms it: "So 233.4 tok/s is the mean, it's a wide interval as you can see from the image", and "In general the dflash head has been mostly optimized for agentic and coding flows, so that's where I'd expect the highest speed-ups." The same reporter who measured 137 tok/s on a general prompt then measured 238.83 tok/s on a coding prompt with acceptance rising from 13.7% to 28.1% — above Meta's published mean, on the same machine and the same build. If your workload is agentic, the headline is reachable; if it is open-ended chat, expect the 137–168 tok/s the first two reporters saw.

The one datapoint on the quant this recipe actually installs is the third row, and it is the slowest of the three — a fair result, since K-Quant-Dynamic is 2.9 GB larger than K-Quant-17GB and was run at the full 131,072-token context. It is also a decode figure with no matching baseline, so it cannot be turned into a speedup. Searching for a complete baseline-vs-DFlash pair on K-Quant-Dynamic, I read all 49 discussion threads on the canonical repo and all 4 on the GGUF repo, and ran a web search for the model plus the card name: none of them carries one. If you run one, please contribute it — it is the single most useful missing number for this page, and it would turn /check/muse-glimmer-30b/rtx-5090 from an empty page into an anchor.

VRAM usage: derived at 22.90 GiB for weights + projector + drafter + KV at full context, as broken down above; not measured on a card by us.

Quality notes: Meta puts K-Quant-Dynamic at 0.2% average degradation across 15 benchmarks against full precision, versus 1.0% for K-Quant-17GB — which is the whole argument for spending a 5090's extra 8 GB on this build. Community reaction in the model's discussions is mixed on the model itself — one thread is titled "Works well, comparable to Qwen 3.6 27B and then some", while a reader in discussion #13 reports the reasoning forgetting and re-iterating. That is a judgement about the model, not about this configuration.

Troubleshooting

unknown model architecture: 'muse-glimmer'

The build predates the architecture. This is the most common report on the GGUF repo, filed against b10338 and reproduced on b10344 (GGUF discussion #1, #2).

Fetching src/llama-arch.cpp at each tag and grepping for LLM_ARCH_MUSE_GLIMMER gives a clean boundary: b10344 resolves and has 0 hits; b10353, b10354 and b10358 resolve and have 1 each. The tags in between — b10345 to b10352 — return HTTP 404, meaning they were never published, not that the feature is missing; the arch commit's own build number, 10349, falls in that unpublished gap, which is why the issue reporter's from-source build says b10349 and no such download exists. Use b10353 or newer, or build from source. Meta's GGUF card independently states the same floor — b10353, with b10344 and older not registering the architecture — which is a useful cross-check on the bisect above rather than a substitute for it. Either way, run the greps in Installation step 2 against your own checkout rather than trusting a build number.

Tool calls after an end-of-message marker get missed

Relevant if you are driving this as an agent, which is the point of the model. PR #26879, "Muse Glimmer: fix detection of tool calls after EOM", merged on 2026-08-11 — a day after the architecture itself. A build cut on 2026-08-10 loads the model perfectly and still mis-parses this case. For agentic use, take a build dated 2026-08-12 or later.

DFlash engages but you see a 1.6× speedup, not 3.1×

Expected on a general-purpose prompt; see Results. One thing not to chase: the GGUF card notes that a [spec] failed to measure draft model memory warning at startup is harmless and the drafter loads and serves normally afterwards. Two things are worth checking before you chase the speedup further. Confirm the startup log shows block_size=16 and a non-zero n_extract — if the drafter loaded as a plain draft model instead, the log will not name draft-dflash at all. And note there is an optimisation in flight that is not in your build. In discussion #21 Meta's ruanslv points the reporter at PR #26842, describing it as "an important optimization, we haven't managed to merge it in time"; the reporter then posts a table in that same thread showing 129.35 tok/s on master against 139.79 with the patch applied. Both of those numbers are from the discussion thread — PR #26842 itself is still a draft, and the benchmark on its page is for different hardware and a different quantisation, so do not read it as a prediction for this card.

--spec-draft-n-max 16 looks accepted but is not

DFlash yields at most block_size - 1 = 15 draft tokens. llama.cpp clamps a larger request and emits requested draft size … exceeds the trained block size 16 -- clamping to 15. Some invocations circulating in the discussions pass 16; they run, they just do not do what they look like they do.

The drafter may refuse to bind against Meta's own GGUF — unresolved

This one is open and genuinely unsettled, and it is filed against the same two files and the same speculative-decoding flags this recipe uses, so read it before you assume a crash is your fault.

Issue #26894 reports that --spec-type draft-dflash crashes at load time with vector::_M_range_check: __n (which is 1) >= this->size() (which is 1) when the target GGUF encodes muse-glimmer.attention.sliding_window_pattern as an array rather than a scalar. The reporter, bordessoules, hit it on a from-source b10349 (62bf73d) CUDA build with muse-glimmer-30B-kquant-dynamic.gguf plus dflash-kquant.gguf, and reports the same drafter binding fine against unsloth's conversion of the same model.

That encoding is in the file this recipe leads with. Range-reading the first 3 MB of each GGUF and walking its metadata block confirms it directly:

So both of Meta's builds carry the reported trigger and the third-party conversion does not. The 12 August republish did not change this: the renamed files carry the same 731 tensors, the same 32 metadata pairs and the same 52-entry array, so nothing in this entry is settled by re-downloading. That is consistent with the report, and it is the reason this entry exists rather than being someone else's problem.

But it does not reproduce for everyone, including for the person who added the architecture. pcuenca, author of the support PR, replied on 12 August: "Hi @bordessoules, I can't reproduce on the current master or 62bf73d." — running the same two files from the same repo. This page corroborates that independently: the K-Quant-Dynamic row in the Results table above is a reader running precisely this configuration on b10358 and getting tokens out rather than a crash.

Four reports actually exercise the drafter, and they do not line up on version.

Every row had to name a draft file to qualify, because --spec-type draft-dflash on its own does nothing: llama.cpp only builds a DFlash speculator when a draft model is actually set, and its constructor asserts on having both a target and a draft context. A posted command carrying the flag but no -md, --model-draft or -hfd is not evidence about the drafter at all — the automatic dflash- sidecar lookup only runs for -hf-style downloads and cannot fill the gap. darkmatter2222 is the lowest build with a startup log confirming the bind, and there is nothing below it. An acceptance rate is the other reliable tell, since a drafter that never bound cannot produce one.

One difference does stand out, and nobody has isolated it. bordessoules is the only reporter running two GPUs — his report names an RTX 3090 alongside an RTX 5060 Ti, built with CMAKE_CUDA_ARCHITECTURES=86;120 to cover both — while all three working runs are on a single device, omaryshchenko pinning his explicitly with device=CUDA0 and spec-draft-device=CUDA0. Version alone cannot carry this, because pcuenca failed to reproduce at 62bf73d, which is build 10349 — the reporter's own. That is an observation about the corpus rather than a diagnosis, but if you hit the crash on a multi-GPU box, restricting llama.cpp to one device is the cheapest thing to vary first.

A merged PR references the issue but does not close it, and does not touch the target path. PR #26900 ("disallow integer dflash sliding_window_pattern", merged 12 August) is a one-line change to src/models/dflash.cpp, swapping get_key_or_arr for get_arr in the drafter's own hparams loader. It does not modify src/models/muse-glimmer.cpp, which is where the target-side key is read, and issue #26894 was still open when this page was written. Note also that the change is newer than every published tag listed above: src/models/dflash.cpp at b10353, b10354 and b10358 still has the old call, so on a release binary you are running the pre-#26900 loader either way.

If you do hit it, the reporter's validated workaround is to rewrite that single metadata key to a scalar with gguf-py's copy_with_new_metadata, setting muse-glimmer.attention.sliding_window_pattern to UINT32 = 4 — 4 being the period of the model's repeating [local, local, local, global] pattern, so it encodes the same layout the 52-entry array spells out. He reports the drafter binding and running afterwards. Running without --spec-type draft-dflash also works and costs you only the speculative speedup; both target files load and generate fine on their own.

Ollama's muse-glimmer:30b is Meta's K-Quant-17GB, repackaged

The library entry exists, takes image input, and labels its default q4_K_M — which reads like a generic community quantisation and is not one. Pulling the registry manifest for :30b and comparing its layer digests against the HuggingFace LFS object ids settles what you actually get:

Range-reading that model layer's own header returns 731 tensors, 32 metadata pairs, general.name = "Muse Glimmer Hf" and the 52-entry sliding_window_pattern array — the same census, name and encoding as Meta's K-Quant-17GB build. So q4_K_M here is Ollama's label on Meta's K-Quant-17GB build, not a separate quantisation. The displayed 18 GB is the sum of the layers (16.76 + 1.40 = 18.16 GB), which is why it matches neither file on its own; that arithmetic is what makes the tag easy to misread.

Two consequences. First, :30b is the 24 GB-target build, so on a 32 GB card it leaves the 0.2%-degradation K-Quant-Dynamic on the table — the whole reason this recipe leads with llama.cpp. Second, :30b-q4_K_M-dflash is a superset, not a different quant: identical projector and model layers plus a 1,631,205,312-byte draft layer whose digest 27d9a805fa29… is bit-for-bit Meta's superseded dflash-kquant.gguf. The plain :30b tag simply has no draft layer, so pulling it and expecting DFlash speedups will disappoint. And because the repack keeps the array encoding, the -dflash tags carry the same trigger condition as the entry above — Ollama is not a way around issue #26894.

One more reason to prefer the first-party path today: those layers were cut before Meta's 12 August republish. The model layer's 16,756,681,056 bytes match the superseded muse-glimmer-30B-kquant-17gb.gguf exactly, and the draft layer's digest is the superseded drafter's, so the Ollama route currently inherits the pre-fix chat template that the GGUF card tells you to re-download to escape. The projector is the one layer unaffected, because that object did not change.