Muse Glimmer 30B on RX 7900 XTX: ROCm llama.cpp with vision and DFlash speculative decoding

What You'll Build

A local llama-server on a single RX 7900 XTX serving Meta's Muse Glimmer 30B agent model over an OpenAI-compatible API — with image input (the perception encoder) and the DFlash drafter for speculative decoding both loaded, at the model's full 131,072-token trained context. Meta ships this model as a GGUF set aimed at a 24 GB envelope; this recipe is the AMD build of that path, because Meta's own instructions only cover CUDA, Metal and CPU.

Hardware data: RX 7900 XTX (24 GB VRAM, gfx1100) · derived working set 20.198 GiB of 24, all three artifacts resident · See benchmark data

⚠️ Nothing here is a speed measurement. Meta published tok/s for an RTX 5090 and two Apple laptops and no AMD figure at all, and /check/muse-glimmer-30b/rx-7900-xtx currently returns verdict: unknown with zero benchmarks. The memory budget below is arithmetic you can re-derive; the throughput is not something this page knows. If you run it, please contribute your numbers.

ℹ️ Do not use the "ROCmFPX" community GGUFs on this card. They are the top AMD-tagged result on the Hub for this model, and they are built for a different chip. Their own card's compatibility section says in as many words that the files will not load in stock llama.cpp and require a patched fork of it, and that they were "targeted and tested on AMD Strix Halo (gfx1151)" — a 128 GB unified-memory APU, not a discrete gfx1100 board. RDNA 3 has no FP4/FP8 matrix hardware, so the format buys you nothing here. Same for the NVFP4 and FP8 quants on the Hub: those are Blackwell/Hopper formats with no RDNA 3 execution path.

Requirements

Why the gfx target matters

llama.cpp's build documentation names your card explicitly: its Windows HIP example uses gfx1100 and says that "corresponds to Radeon RX 7900XTX/XT/GRE". AMD's ROCm system-requirements matrix lists the same card as officially supported, which is the part that matters operationally: you do not need HSA_OVERRIDE_GFX_VERSION. That variable is a masquerade for cards ROCm does not ship kernels for. Setting it on a supported gfx1100 board only makes your logs lie about what ran.

Installation

1. Confirm ROCm sees the card

rocminfo | grep -m1 gfx
# expect: gfx1100

2. Build llama.cpp for HIP / gfx1100

Meta's GGUF card gives a CUDA build line. Replace it — -DGGML_CUDA=ON has no meaning on this hardware. The HIP form comes from llama.cpp's own build docs, with GPU_TARGETS set to your card's target instead of the doc's gfx1030 example:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
  cmake -S . -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1100 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j 16 \
  --target llama-cli llama-mtmd-cli llama-server

Muse Glimmer support landed in PR #26841, merged to master on 10 Aug 2026 as commit 62bf73d, and first shipped in release b10353. Verify your checkout before you download 20 GB of weights:

grep -c LLM_ARCH_MUSE_GLIMMER src/llama-arch.cpp   # expect 1, not 0
./build/bin/llama-server --version                 # build number >= 10353

3. Download the weights

Three files: the 24 GB-targeted text build, the perception encoder, and the drafter. Meta's card is blunt that the last two are add-ons — "Both text builds are text-only on their own; the two companions are additive."

pip install huggingface_hub

hf download meta-models/Muse-Glimmer-30B-GGUF \
    --local-dir Muse-Glimmer-30B-GGUF \
    --include "Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf" \
    --include "mmproj-Muse-Glimmer-30B-Q4_K_M.gguf" \
    --include "dflash-Muse-Glimmer-30B-Q4_K_M.gguf"

⚠️ Use those exact names, and re-pull if you downloaded earlier. On 12 Aug 2026 Meta republished the text build and drafter under these canonical Q4_K names with a corrected embedded chat template; the card says of the old ones that "Earlier GGUFs shipped a template that skipped both, which could leave a conflicting Reasoning strength: high. in the prompt — re-download if you pulled before this fix." The superseded copies are still sitting in the repo under their old names, so a stale command line will succeed and quietly give you the defective template. The new files are 2,848 bytes larger (2,816 for the drafter) — the longer template, nothing else: I re-read both headers and they are byte-for-byte the same model, 731 tensors, sliding_window_pattern still a 52-entry array. The projector was not reissued; mmproj-Muse-Glimmer-30B-Q4_K_M.gguf and the old mmproj-kquant.gguf share one object id, so only the name changed there.

Running

./build/bin/llama-server \
    -m       Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf \
    --mmproj Muse-Glimmer-30B-GGUF/mmproj-Muse-Glimmer-30B-Q4_K_M.gguf \
    -md      Muse-Glimmer-30B-GGUF/dflash-Muse-Glimmer-30B-Q4_K_M.gguf \
    --spec-type draft-dflash -ngld all \
    -a muse-glimmer-30B \
    -ngl 99 -c 131072 -np 1 \
    --host 127.0.0.1 --port 8080 \
    --jinja \
    --temp 1.0 --top-p 0.95 --top-k 64

-np 1 is deliberate. llama-server divides -c across its slots, so with the vendor's -np 4 a single request would only get 32,768 tokens; with one slot the whole 131,072 belongs to it. Read n_ctx_seq in the startup log rather than trusting -c. --jinja is not optional — the chat template is embedded in the GGUF and llama-mtmd-cli refuses to start without it. Leave -fa alone: it defaults to auto, and llama.cpp decides whether its AMD WMMA flash-attention kernels apply on your build; the resolved value is printed as flash_attn at startup.

Sanity-check the API:

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"muse-glimmer-30B",
       "messages":[{"role":"user","content":"What is 17 * 23? Reply with just the number."}]}' \
| python3 -c "import json,sys; m=json.load(sys.stdin)['choices'][0]['message']; \
print('content  :', m['content']); print('reasoning:', len(m.get('reasoning_content') or ''), 'chars')"

For a one-shot image on the command line, use llama-mtmd-cli, not llama-cli:

./build/bin/llama-mtmd-cli \
    -m       Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf \
    --mmproj Muse-Glimmer-30B-GGUF/mmproj-Muse-Glimmer-30B-Q4_K_M.gguf \
    -ngl 99 -c 32768 --jinja \
    --temp 1.0 --top-p 0.95 --top-k 64 \
    --image photo.png -p "Describe this image."

Watch the load log for CLIP using ROCm0 backend. That line is llama.cpp's own report of where the vision graph landed; if it says CLIP using CPU backend, image encoding fell back to the host and you should read the accompanying "CLIP graph uses unsupported operators by the backend" warning before going further.

One-command alternative: Ollama

Ollama officially supports RDNA 3, and its muse-glimmer:30b tag skips the compile entirely:

ollama run muse-glimmer:30b

Its tags page lists that tag at 18 GB with a 128K window and image input, and the registry manifest shows exactly why: a model layer of 16,756,681,056 bytes — the byte count of Meta's muse-glimmer-30B-kquant-17gb.gguf — plus a projector layer whose SHA-256 (f48b4523…) is that of Meta's mmproj-kquant.gguf. So it is the same 24 GB-targeted build with the same encoder.

The :30b tag has no drafter layer, so it runs without DFlash — but you do not have to give that up to use Ollama. The 30b-q4_K_M-dflash manifest is that same pair plus one: byte-identical model and projector layers (same digests, same sizes) and a third layer of 1,631,205,312 bytes whose SHA-256 27d9a805fa29b943cfb6ad4843367cd4eaaaf06bd452d8cc3e00a2cd18a677bc is exactly the LFS object id of Meta's dflash-kquant.gguf. Despite the q4_K_M in its name it is not a different quantisation of the model — only the 64-byte params blob differs from the :30b tag's 42-byte one:

ollama run muse-glimmer:30b-q4_K_M-dflash

Both digests above are the pre-fix objects, and that is the catch: Ollama's layers are byte-identical to the GGUFs Meta superseded on 12 Aug, so this route ships the old chat template that the vendor now tells you to re-download away from. Until Ollama repulls, the llama.cpp path above is the one that gets the corrected template.

What Ollama does not get you is an escape from the drafter bug in Troubleshooting below. Range-reading the head of Ollama's own model blob and parsing the GGUF header gives 731 tensors and muse-glimmer.attention.sliding_window_pattern as the same 52-entry boolean array Meta ships. The key sits in the first 128 KB, so the cheap check needs no parser:

curl -sL -H 'Range: bytes=0-131071' \
  https://registry.ollama.ai/v2/library/muse-glimmer/blobs/sha256:71b5c9c9abbc2a9e7c13550b6f9fe1f062650117a24490b15a310cbd7de03f8b \
  | grep -a -o -m1 'muse-glimmer.attention.sliding_window_pattern'

The -L matters: the registry answers with a 307 to its CDN, and without it you get an empty body and a silent miss.

Results

• Speed: omitted. Meta's speed table covers an RTX 5090 and the Apple M4/M5 Max only — there is no AMD row in it — and searching the discussion threads on meta-models/Muse-Glimmer-30B, meta-models/Muse-Glimmer-30B-GGUF, unsloth/Muse-Glimmer-30B-GGUF and bartowski/Muse-Glimmer-30B-GGUF (70 threads, fetched individually rather than sampled) plus the llama.cpp issue tracker turned up no third-party tok/s figure for this card. Publishing a borrowed number would be inventing one. Send yours and it becomes the first.

• It runs, and that part is not derived. A community user reports this exact three-file configuration working on this exact card in discussion #2 on Meta's GGUF repo: they post a llama-server command with -mm mmproj-kquant.gguf -md dflash-kquant.gguf --spec-type draft-dflash, say it is "now working well for me after I built the latest release of llama.cpp as of about 5 hours ago" on build 10358, and sign off with the hardware — "24G 7900 XTX ROCm". Community report — no measurement and no startup log attached — and the only gfx1100 datapoint in any of those 70 threads.

• Vision is a real mainline path, not an inference from the file's existence. The mmproj-Muse-Glimmer-30B-Q4_K_M.gguf header declares clip.projector_type = "muse-glimmer"; llama.cpp's tools/mtmd/clip-impl.h maps that exact string to PROJECTOR_TYPE_MUSE_GLIMMER, and PR #26841 added tools/mtmd/models/muse-glimmer.cpp to build its graph. Nothing in that path is CUDA-specific: clip.cpp contains zero cuda/hip/rocm/GGML_USE_* matches and picks its device with a generic ggml_backend_init_by_type(GGML_BACKEND_DEVICE_TYPE_GPU), which is why the same code runs under HIP.

• VRAM usage — the derivation. Every term is a byte count from the HF tree API or from llama.cpp's own cache sizing, so you can redo it:

  Component	GiB
  Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf (16,756,683,904 B)	15.606
  mmproj-Muse-Glimmer-30B-Q4_K_M.gguf (1,400,328,928 B)	1.304
  dflash-Muse-Glimmer-30B-Q4_K_M.gguf (1,631,208,128 B)	1.519
  KV, 13 full-attention layers × 131,072 tokens, f16	1.625
  KV, 39 sliding-window layers × 2,560 cells, f16	0.095
  Drafter KV, 5 sliding layers × 2,560 cells, f16	0.049
  Total	20.198

  The KV rows are why the full context is affordable. The GGUF's sliding_window_pattern key is a 52-entry boolean array in the repeating shape [sliding, sliding, sliding, full], so 13 layers cache the whole context and 39 cache only a window. With 2 KV heads at head dimension 128, one token costs 1,024 bytes per layer at f16. llama.cpp sizes the windowed half as GGML_PAD(min(n_ctx, n_swa × n_seq_max + n_ubatch), 256) — 2,048 + 512 rounded up gives 2,560 cells, not 131,072. Meta's card makes the same point in prose: the cache costs a few GB, not tens. That leaves 3.802 GiB of the card's 24 for ggml's compute buffers, the image-encoding working set, and whatever your desktop is using — run headless or on the integrated display output if you want all of it.

For the full benchmark data, see /check/muse-glimmer-30b/rx-7900-xtx.

Troubleshooting

unknown model architecture: 'muse-glimmer'

By far the most-reported error, and it is always build vintage. Support merged on 10 Aug 2026; releases b10344 and older do not register the architecture. Reports against Vulkan builds (#26858, #26865) look like a backend problem and are not: #26865's reporter says in the same breath that "The Latest build with ROCm does work." — on their AMD machine the ROCm binary was current and the Vulkan one was built from an older commit. A user on discussion #1 on Meta's GGUF repo answers the same question directly: a from-source build reporting version: 10352 — a number no release carries — loads kquant-dynamic without complaint. Run grep -c LLM_ARCH_MUSE_GLIMMER src/llama-arch.cpp on your checkout: 0 means the source predates support and no runtime flag will help.

The drafter crashes at load with vector::_M_range_check

Issue #26894 reports that binding the DFlash drafter aborts when the target GGUF encodes muse-glimmer.attention.sliding_window_pattern as an array — and the file this recipe uses does exactly that, as does its 32 GB sibling and the blob Ollama serves, so no runtime choice above dodges it. Read the issue before you panic, though: the reporter's own finding is that the same drafter binds fine against a third-party conversion of the same model that encodes the key as a scalar, and their validated fix is to rewrite that one metadata key to a scalar UINT32 = 4 with gguf-py's copy_with_new_metadata. The issue is unconfirmed: on 12 Aug 2026 the author of the Muse Glimmer support PR replied that he could not reproduce it on master or on 62bf73d with the plain -m … -md … --spec-type draft-dflash invocation, and the community RX 7900 XTX report cited above ran the same three files on b10358 without hitting it.

The one failure carries a confound, and it is not the build number. pcuenca could not reproduce on 62bf73d — which is b10349, the reporter's own build — so build vintage cannot explain both facts. What separates them is the environment: #26894 is the only multi-GPU report in the corpus, an RTX 3090 alongside an RTX 5060 Ti in a Docker container on a Windows host, compiled dual-arch with CMAKE_CUDA_ARCHITECTURES=86;120. Nobody has run that configuration on one GPU, so the confound is unisolated — an open question, not a verdict. The sharpest counterpoint is a run that pinned a single device explicitly: the b10358 config below sets device=CUDA0 and spec-draft-device=CUDA0, target and drafter both, against the very file the issue was filed on, and it works.

Against it are three single-GPU runs that actually loaded a drafter, and they are not the same strength of evidence. Two show it demonstrably bound, both RTX 5090 reports in discussion #34 on Meta's base repo: b10354 on kquant-17gb with --flash-attn on, whose pasted startup log contains adding speculative implementation 'draft-dflash'; and b10358 on kquant-dynamic — the very file the issue names — posted as a complete llama.cpp INI config that names both halves (model and md) and pins both devices, with a measured 25% draft acceptance rate corroborating it, a figure an unbound drafter cannot emit. That second one is arguably the strongest datapoint in the corpus: full configuration, filed-against file, single-device pin, and a metric. The third is this recipe's own card and it is the weakest of the three: the b10358 RX 7900 XTX run cited above names the drafter with -md and --spec-type draft-dflash and says it works, but pastes no startup log and no acceptance figure, so the bind is a fair inference rather than a measurement. b10354 is therefore the lowest log-confirmed build.

On a single RX 7900 XTX you should not expect to hit this, but the evidence for your card specifically is a user's word rather than a log. Keep the metadata rewrite in reserve — the issue is open and its mechanism is understood — and change nothing before you actually see the abort.

[spec] failed to measure draft model memory at startup

Harmless. Meta's card says of the drafter flags: "Same outputs, faster decode, ~1.6 GB extra." — the warning fires during memory fitting and the draft model loads and serves normally afterwards. Do not remove -md because of it.

Memory grows and prompt processing slows after the first image

Issue #26873 (open, unconfirmed) reports that after the first use of the projector the GPU holding it consumes 1.1 GB beyond the encoder's own weights, and prefill throughput drops by about a third and stays down even on a fresh context. It was filed on CUDA hardware, so treat it as a caution rather than a gfx1100 finding — but it is spending part of your 3.802 GiB, and it is the reason to run headless if you are close to the edge. Restarting the server clears it.

Empty replies on long generations

Nothing errors when a generation runs out of context — the request simply returns no answer. This model reasons at length, so it hits that wall sooner than most. Check n_ctx_slot in the startup log: with -np 1 it should read 131072. If you raised -np, raise -c by the same factor.

Should you use the Vulkan backend instead?

Possibly, but do not assume it. Issue #20934 (closed) benchmarked this exact card and found ROCm generating tokens materially slower than Vulkan — on 7B-class GGUFs, on March 2026 builds, not on this model. Meanwhile the Vulkan binaries were the last to carry Muse Glimmer support. If you want the comparison, build both and run llama-bench yourself; that is a measurement worth sending in.