Muse Glimmer 30B on RTX 3090: vision + DFlash speculation at full 131K context

What You'll Build

A local agent endpoint on a single RTX 3090: llama-server running Meta's Muse Glimmer 30B with image input and DFlash speculative decoding, at the model's full 131,072-token context — not a reduced one. Every component is resident on the card; nothing offloads to system RAM.

Hardware data: RTX 3090 (24 GB VRAM) · derived resident set 20.198 GiB at 131,072 context, 3.802 GiB spare · See benchmark data

⚠️ Known issue: binding the DFlash drafter to Meta's own GGUF is the subject of an open llama.cpp bug, #26894. The evidence is lopsided: one crash, reported from a two-GPU Docker build and not reproducible by the support PR's own author on that same commit, against three independent single-GPU runs that bind the drafter fine. See Troubleshooting before you add --spec-type.

ℹ️ Multimodal means text + images, not audio. Muse Glimmer accepts interleaved text and images through a ViT-G/14 perception encoder, and emits text only. The model card is explicit: "Audio input/output is not supported." Video is not a supported input either — the card notes the model is not explicitly optimized for it and processes it as individual frames.

Requirements

Ampere has no FP4 or FP8 tensor path, so the NVFP4 and MXFP8 builds Meta ships for other targets are not options here. On an RTX 3090 the route is the K-quant GGUF stack below.

Installation

1. Get a llama.cpp build that knows the architecture

Muse Glimmer support landed in PR #26841, merged 2026-08-10 as commit 62bf73d2. Release b10353 is the first tag that contains it — b10344 and earlier sit five commits behind the merge and fail with unknown model architecture: 'muse-glimmer'.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout b10358
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86
cmake --build build --config Release -j

CMAKE_CUDA_ARCHITECTURES=86 is the RTX 3090's compute capability (Ampere, sm_86). Prebuilt release binaries from the same tag work equally well if you prefer not to compile.

2. Download the three GGUF files

hf download meta-models/Muse-Glimmer-30B-GGUF \
  Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf \
  mmproj-Muse-Glimmer-30B-Q4_K_M.gguf \
  dflash-Muse-Glimmer-30B-Q4_K_M.gguf \
  --local-dir ./muse-glimmer

Three files, three jobs: the 16.76 GB quantised language model, the 1.40 GB vision projector, and the 1.63 GB DFlash drafter. Meta's model card describes the K-Quant-17GB build as compressing weights to roughly 4-bit, "shrinking the language model to under 20 GB", and states that this "leaves enough headroom for the model's KV cache, the perception encoder for image understanding, and the speculative decoding drafter to run simultaneously within a 24 GB or 32 GB envelope." The budget under Results checks that claim against the actual bytes.

⚠️ Use the canonical Q4_K_M names above, not the older *-kquant*.gguf ones. Meta republished these builds on 2026-08-12 with a corrected embedded chat template; the previous template could leave a conflicting reasoning-strength line in the prompt, and the GGUF card says to "re-download if you pulled before this fix". Both sets are still in the repo, so a stale filename silently fetches the superseded build. The projector is the exception — mmproj-Muse-Glimmer-30B-Q4_K_M.gguf is the same object as the old mmproj-kquant.gguf (identical SHA-256 f48b4523…), so there is nothing to re-download there.

Running

./build/bin/llama-server \
  -m ./muse-glimmer/Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf \
  --mmproj ./muse-glimmer/mmproj-Muse-Glimmer-30B-Q4_K_M.gguf \
  -ngl 99 -c 131072 -np 1 \
  --jinja \
  --host 127.0.0.1 --port 8080

This is the shape Meta publishes in its own llama.cpp deployment doc, which also names b10353 as the minimum release. --jinja is not optional: the model ships an ATEM tool-calling chat template, and without --jinja tool calls and reasoning separation break. -np 1 matters for the memory budget — each extra server slot multiplies the sliding-window KV ring.

To add speculative decoding, append the drafter:

  -md ./muse-glimmer/dflash-Muse-Glimmer-30B-Q4_K_M.gguf \
  --spec-type draft-dflash -ngld 99 --spec-draft-n-max 4

Read Troubleshooting first — this is the flag pair that trips open issue #26894 on some builds.

Meta's recommended sampling is --temp 1.0 --top-k 64 --top-p 0.95. Reasoning strength is set in the system prompt as Reasoning strength: <low|medium|high|xhigh>, or through --chat-template-kwargs '{"reasoning_strength":"low"}'.

Ollama, if you would rather not build anything

ollama run muse-glimmer:30b pulls the same build — though not, for one of the two layers, the same file. Its registry manifest lists layers of 16,756,681,056 and 1,400,328,928 bytes, and the distinction matters:

• The projector layer is bit-identical to Meta's projector — same SHA-256, f48b4523…, which is both mmproj-kquant.gguf and its renamed twin.
• The model layer matches muse-glimmer-30B-kquant-17gb.gguf to the byte in size, but its content hash differs (71b5c9c9… against Meta's 7e9b74b7…). It is the same K-Quant-17GB build, re-packaged, not the same bytes.

Ollama labels the tag q4_K_M and displays 18 GB, but this is the 24 GB-target build — not the 32 GB-target K-Quant-Dynamic one.

One caveat as of 2026-08-12: Ollama's blobs still correspond to the pre-fix files, so this route carries the older chat template that the republish above corrects. If you rely on reasoning-strength control, prefer the llama.cpp path with the canonical files.

For the drafter as well, pull muse-glimmer:30b-q4_K_M-dflash: the same two layers plus a third of exactly 1,631,205,312 bytes at digest 27d9a805…, which is bit-identical to Meta's dflash-kquant.gguf.

Results

• Speed: no first-party RTX 3090 figure exists. Meta's speed table publishes an RTX 5090, an Apple M4 Max and an M5 Max and nothing else — the string "3090" does not appear anywhere in the model card. Community reports on a single 3090 are collected below; they are single-source each and disagree, so treat them as orientation, not as a benchmark. If you measure this pair, please contribute the numbers.
• VRAM usage: 20.198 GiB derived resident set at full context — see the budget below and /check/muse-glimmer-30b/rtx-3090.
• Quality notes: the model card rates the K-Quant-17GB build at 1.0% average degradation across 15 benchmarks, against 0.2% for the 32 GB-target K-Quant-Dynamic build.

The 24 GB budget

The interesting question on a 24 GB Ampere card is not whether 16.76 GB of weights fit — it is whether they still fit once the vision projector, the drafter and a 131,072-token KV cache sit beside them. They do, with room to spare, and the reason is architectural.

Muse Glimmer uses grouped-query attention with 32 query heads against 2 KV heads and a 128-wide head, so one token costs just 1,024 bytes of f16 KV per layer. On top of that, its attention pattern is [Local, Local, Local, Global] repeating — which the GGUF encodes as a 52-entry sliding_window_pattern of 39 sliding-window layers and 13 full-attention layers. llama.cpp allocates these as two separate caches (build_attn_inp_kv_iswa): only the 13 full-attention layers scale with context, while the 39 sliding-window layers are pinned to a small ring of sliding_window + n_ubatch cells — 2,560 at the default -ub 512. So 75% of the layers cost nothing as context grows.

That leaves 3.802 GiB of the card's 24 GiB. Two things live in that margin and are not in the table, because we did not measure them: llama.cpp's CUDA compute buffer, which scales with -ub, and the CUDA context plus whatever your desktop is holding. Budget accordingly on a machine driving a monitor.

One cross-check says the arithmetic is sound: running the same derivation on the K-Quant-Dynamic build gives 22.897 GiB — only 1.103 GiB clear, which is why Meta targets that build at 32 GB rather than 24 GB. Our numbers reproduce the vendor's own two-tier split without being fitted to it.

Community RTX 3090 reports

None of these are ours, and none are first-party. Each names its own configuration, and the configurations differ more than the numbers do.

The closest match to this recipe is the first: "Just tried the gguf 17gb k-quant with llama.cpp" — the same file this recipe installs — reporting that "when using the dflash drafter, it does more than 60 tok/sec on my 3090" and that "vision (with the additional mmproj model) seems to work really well". That is one community datapoint from one person, not a measurement, and the same commenter was unimpressed with the model's output quality.

For the full benchmark data, see /check/muse-glimmer-30b/rtx-3090.

Troubleshooting

unknown model architecture: 'muse-glimmer'

Your llama.cpp predates the architecture. Release b10344 and earlier are five commits behind the #26841 merge; upgrade to b10353 or newer. Users on the official GGUF repo hit this on b10344 and resolved it by moving to b10352 and b10358.

The DFlash drafter crashes with vector::_M_range_check

vector::_M_range_check: __n (which is 1) >= this->size() (which is 1)

Open issue #26894 reports that the DFlash bind path fails when the target GGUF stores muse-glimmer.attention.sliding_window_pattern as an array rather than a scalar. We read that key out of the file this recipe installs: Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf does store it as a 52-entry array — and so did the build it superseded, so the 2026-08-12 republish does not change this. The unsloth conversions store it as scalar 4 instead.

One crash report, three independent successes, and the difference is not the thing it looks like. Every public report that actually loads a drafter file:

Two candidate explanations are already ruled out. The encoding is not the discriminator: every target above is array-encoded, including the last row — the same file the crash was filed against. Flash-attention is not it either: the b10354 run passes --flash-attn on and binds cleanly.

Nor is it cleanly a version story, because pcuenca could not reproduce on 62bf73d — the reporter's own commit. The one factor nobody has isolated is that bordessoules is the only reporter running two GPUs, in Docker; every success is single-GPU. Treat that as unexplained rather than settled.

A fourth report, dmpr, runs K-Quant-Dynamic at 131,072 context on build 10352 and calls it working — but that thread is about the model failing to load, and the command as posted names no draft model file. llama.cpp only auto-discovers a dflash- sidecar for -hf downloads, so with a local --model path and no -md there is no draft context to bind. Read it as evidence that the architecture loads at 10352, not that the drafter binds there.

PR #26900 was merged on 2026-08-12, but its author struck through the claim that it closes #26894 and the issue remains open — its diff touches only src/models/dflash.cpp, swapping get_key_or_arr for get_arr in the drafter's own metadata read, while the target-side read in src/models/muse-glimmer.cpp still uses get_key_or_arr. That is both why the strikethrough is there and why the unsloth-target workaround below survives the merge.

Ollama is not an escape hatch from this. We read the GGUF header straight out of Ollama's own model blob over an HTTP range request: the repack retains the same 52-entry array, so a user on the -dflash tag family is exposed to exactly the bug a llama.cpp user pointed at Meta's file is.

The real escape hatches, in ascending order of effort: drop --spec-type draft-dflash and run without speculation (the model still serves at full context — subtract 1.519 GiB and 0.049 GiB from the budget); or point -m at an unsloth build, whose scalar encoding the reporter confirmed binds with the same unmodified drafter file.

VRAM climbs after the first image, and prefill gets slower

Community issue #26873 reports that the first use of the vision projector costs a further 1.1 GB beyond the mmproj weights already accounted for, and that prompt-processing throughput then falls from 3,062 to 1,875 tokens per second in the reporter's own logs — and stays down even on a fresh context. It is an open report with no maintainer response yet, measured on different hardware and the K-Quant-Dynamic build, so the exact figures may not transfer. It matters here because it eats into the 3.802 GiB margin: if you serve images and run close to the edge, lower -c or drop the drafter.

DSpark is not DFlash, and it can be slower

Meta ships a DFlash drafter; llama.cpp also supports a DSpark type, which adds a Markov head and outranks DFlash in llama.cpp's sidecar auto-detection when both are present. They are not interchangeable. On a single RTX 3090 bartowski measured his own Q4_K_M build at "On my 3090 with Q4_K_M I get around 38 tok/s, with dspark it drops to ~28-32 depending on the task". Pass --spec-type draft-dflash explicitly rather than relying on auto-detection.

-hf picks the wrong file

llama.cpp's repo auto-selection looks for Q4_K_M or Q8_0 in the filename, then falls back to the first model-shaped GGUF in the repo. Since the 2026-08-12 republish the canonical name carries Q4_K_M, so a bare -hf meta-models/Muse-Glimmer-30B-GGUF now resolves to Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf by a real tag match rather than by ordering. The superseded *-kquant*.gguf files are still in the repo and match neither pattern, so if you are pinning a name, pin the canonical one: --hf-file Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf.