Qwen3.8-27B on RTX 4060 Ti 16GB: a vision-capable 27B on 288 GB/s

What You'll Build

A local OpenAI-compatible server running Qwen3.8-27B — a 27B dense vision-language model — on a single RTX 4060 Ti 16GB, with the vision projector loaded and a 32,768-token context. The install leads with llama-server from llama.cpp, a 3-bit dynamic GGUF from unsloth, and an 8-bit KV cache.

Hardware data: RTX 4060 Ti 16GB (16 GB VRAM, 128-bit GDDR6, 288 GB/s) · derived working set 14.590 GiB of 16 GiB · See benchmark data

⚠️ Nothing on this page was measured on an RTX 4060 Ti. /check/qwen3-8-27b/rtx-4060-ti-16gb returns verdict: unknown with zero benchmarks, and no measurement of this model on this card surfaced anywhere this research reached. Every number below is either a byte count read from the artifact you download, arithmetic derived from llama.cpp's own source, or a clearly-labelled figure for a different model on this same card. If you run this, please contribute your numbers — that is how the /check page gets real data.

ℹ️ This card sits at the honest edge of the 16 GB tier, and the reason is the bus, not the capacity. The model fits — the arithmetic under VRAM budget is the same on every 16 GB card. What is not the same is how fast the card can read those bytes back. The RTX 4060 Ti 16GB pairs its 16 GB with a 128-bit GDDR6 bus at 288 GB/s, the narrowest in this tier, and a 27B at 3 bits is bandwidth-bound the moment it starts generating. What the 288 GB/s bus costs you sets the expectation with numbers rather than adjectives.

Requirements

Why a 27B fits at all: the architecture

Qwen3.8-27B is not a conventional 64-layer dense transformer, and that is the whole reason a 16 GB card is even in the conversation. Its config.json declares 64 layers of which 48 are linear_attention and 16 are full_attention, at indices 3, 7, 11 … 63 — the model card describes the pattern as "16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN))".

llama.cpp implements this as architecture qwen35 and derives the same split independently. In src/models/qwen35.cpp it marks recurrent layers as is_recr_impl[i] = (i < hparams.n_layer()) && ((i + 1) % full_attn_interval != 0), with full_attn_interval defaulting to 4 and read from the GGUF key qwen35.full_attention_interval. Layers 3, 7, 11 … 63 are therefore the only ones that get a KV cache; src/llama-model.cpp confirms this with a layer filter of il < hparams.n_layer() && !hparams.is_recr(il) when it builds the hybrid memory.

The consequence: only 16 layers hold a KV cache instead of 64, so KV costs a quarter of what a conventional 27B would charge. The other 48 layers hold a fixed-size recurrent state that does not grow with context at all.

Count the layers from 64, not from the GGUF's block_count. Publishers disagree here: ggml-org's files declare block_count = 64 and ship the multi-token-prediction head as a separate mtp-*.gguf, while unsloth, bartowski and lmstudio-community declare 65 with nextn_predict_layers = 1 and the head inline. Both describe the same model. llama.cpp reconciles them in src/llama-hparams.cpp with n_layer() { return n_layer_all - n_layer_nextn; } — commented in the header as "number of effective layers (excludes nextn layers)" — and the MTP block is excluded from the attention cache by the same il < hparams.n_layer() test. Sizing KV off a raw block_count of 65 would give 16.25 layers and overstate the cache by about 6%.

VRAM budget

Every figure here is either a byte count from the HuggingFace tree API, re-read for this recipe rather than inherited, or arithmetic over values in config.json and llama.cpp's source.

On units, because this is where fit calculations go wrong. Everything below is GiB (2^30 bytes) on both sides of the comparison. A "16 GB" graphics card is 16 GiB = 16,384 MiB — GPU memory is quoted in binary units, unlike disk capacity, and the whole stack agrees: ggml-cuda.cu prints device memory as prop.totalGlobalMem / (1024 * 1024) labelled MiB, and nvidia-smi reports the same. Do not convert the card's "16 GB" as if it were a decimal figure — that would understate it by 7% and turn a fit into a miss. Two real deductions do apply and both come out of the headroom line rather than the components: cudaDeviceProp::totalGlobalMem sits slightly under the nominal 16,384 MiB, and the display driver reserves more on top — noticeably more on Windows with a monitor attached than on a headless Linux box. Check yours with nvidia-smi --query-gpu=memory.total,memory.used --format=csv before you size the context.

KV cache, per token. 16 full-attention layers × 4 KV heads × 256 head dim × 2 (K and V) = 32,768 elements per token. config.json gives num_key_value_heads = 4 and head_dim = 256, and the GGUF header carries the same shape as qwen35.attention.head_count_kv, key_length and value_length. At fp16 that is 65,536 bytes per token; at q8_0 (34 bytes per 32-element block) it is 34,816 bytes per token.

Recurrent state, per sequence. llama_hparams::n_embd_r() returns (ssm_d_conv - 1) * (ssm_d_inner + 2 * ssm_n_group * ssm_d_state) = 3 × (6144 + 2×16×128) = 30,720 elements of convolution state, and n_embd_s() returns ssm_d_state * ssm_d_inner = 128 × 6144 = 786,432 elements of recurrent state, per layer. Those inputs come straight out of config.json (linear_num_value_heads 48 × linear_value_head_dim 128 = 6144; linear_num_key_heads 16; linear_key_head_dim 128; linear_conv_kernel_dim 4). Both are allocated as GGML_TYPE_F32. Across 48 layers for one sequence that is 48 × (30,720 + 786,432) × 4 = 156,893,184 bytes, and it is constant regardless of context length.

The whole budget at 32,768 context, --parallel 1, q8_0 KV:

That 1.410 GiB is what the CUDA context, the compute buffers, the vision encoder graph and the display driver's own reservation have to live in. It is a reservation, not a measurement — llama.cpp allocates those from the same pool and their size depends on your batch and image sizes, and none of it was measured on this card. Treat 32K as a starting point to verify rather than a guarantee. If your run overflows, the ladder under Troubleshooting walks you down.

One thing works in your favour that the table does not credit: llama.cpp skips the model's multi-token-prediction block unless you actually ask for speculation. qwen35.cpp creates those tensors with int mtp_flags = !ml.load_mtp ? TENSOR_SKIP : 0;, so the extra blk.64 carried inside the unsloth, bartowski and lmstudio-community files is on disk but not in VRAM by default.

What the 288 GB/s bus costs you

This is the section that makes an RTX 4060 Ti 16GB page different from a page about any other 16 GB card, so it gets numbers rather than adjectives.

The specs. NVIDIA's own product page lists the RTX 4060 Ti as 16 GB GDDR6 on a 128-bit interface; it does not publish a bandwidth figure, and Hardware Corner's RTX 4060 Ti 16GB page states 288 GB/s against that same 128-bit bus. That is the floor of its capacity class. Among the NVIDIA cards this catalogue holds at 16 GB — RTX 4060 Ti 16GB, RTX 4070 Ti Super, RTX 4080, RTX 4080 Super, RTX 5060 Ti, RTX 5070 Ti and RTX 5080 — the fastest is the RTX 5080 at 960 GB/s, with the RTX 5070 Ti at 896 and the RTX 5060 Ti at 448. (The tier also contains an Apple M2 Pro and a Radeon RX 7800 XT, which this page does not characterise.) Identical capacity, a 3.33× spread in bandwidth — and this card is the slow end of it.

Why that matters more here than for most models. Generating a token requires reading the whole weight set — 13,441,059,904 bytes at this quant — plus whatever KV has accumulated. Prompt processing does not: it reads the weights once and then does arithmetic over many tokens at a time, so it is limited by compute rather than by memory. A narrow bus therefore taxes one half of the workload and mostly spares the other.

That split is measurable across the tier. The figures below are Hardware Corner's published tables for Qwen3 14B at Q4_K at a 32K context — a different, smaller model, quoted as a hardware reference point and not as a figure for Qwen3.8-27B:

At that 32K column, decode on this card runs at 0.34× the fastest NVIDIA card at this capacity, and prefill at 0.41×. The sharper illustration is the same source's 4K column against the RTX 5060 Ti — the same 128-bit width, newer memory: 27.4 against 41.1 tok/s of decode (0.67×), but 1,645.7 against 1,743.0 t/s of prompt processing (0.94×). Nearly the same prompt speed, two-thirds the generation speed. That is the whole shape of the trade: the narrow bus costs you the answer, not the reading of the question.

One row in that table is worth not glossing over: at 32K the RTX 5080's prefill (1,326.1 t/s) comes in below the RTX 5070 Ti's (1,658.2), even though it has more bandwidth — and the two swap back at 4K and 16K in Hardware Corner's fuller tables. Decode, by contrast, rises monotonically with bandwidth down the whole column. That is the same point from the other direction: generation tracks memory bandwidth closely enough to rank cards by it, and prompt processing does not, so a prefill number is not something you should extrapolate from a bandwidth figure.

Those two 4K figures are the ones our own catalogue carries for this card — 27.4 tok/s and 1,645.7 t/s, at /check/qwen3-14b/rtx-4060-ti-16gb. The 32K column above is Hardware Corner's alone; we hold no 32K row for it.

The ceiling for this model, as arithmetic. Divide the bandwidth by the bytes that must be read per generated token. This is reasoning, not measurement — it is an upper bound that no real runtime reaches, because it credits the card with perfect memory efficiency and zero time spent on anything else:

• with an empty context, 288 GB/s ÷ 13.441 GB of weights ≈ 21 tokens/s
• with a full 32K window, 288 GB/s ÷ 14.739 GB (weights + KV + recurrent state) ≈ 20 tokens/s

Expect to land materially under both. The honest summary is that this configuration is for reading, looking at images and thinking, not for watching text stream. A community user on an unnamed 16 GB card put the same conclusion less kindly in discussion #54 — Qwen3.8-27B is "not really usable with that speed on a 16gb vram". Their post names no card, no quant and no runtime, and its numbers are a comparison against a different model, so it is sentiment rather than a measurement; it is quoted here because it is the closest thing to first-hand 16 GB experience that this research found, and because it agrees with the arithmetic.

If you run this configuration, the one thing that would improve this page is your llama-bench output via /contribute.

Picking a quant on 16 GB

There is no first-party GGUF. Qwen publishes Qwen3.8-27B and an FP8 sibling as safetensors only; filtering their org's model list to "Qwen3.8" returns exactly four repositories and none of them is a -GGUF repo, while the same org's GGUF-filtered listing returns 54 repos for other model families. Every GGUF below is a community conversion, so the publisher is part of what you are choosing.

Sizes below are size fields from https://huggingface.co/api/models/<repo>/tree/main?recursive=true, converted to GiB, re-read for this recipe.

Scope of this survey, so you know what it does and does not cover. HuggingFace lists over 200 GGUF repositories matching "Qwen3.8-27B" — abliterated forks, distills, ROCm/NVFP4 repacks, and one publisher shipping 90+ single-quant split repos. Enumerating that tail is not feasible and most of it is not this model. What follows is every 4-bit build from the five publishers this recipe surveys — ggml-org, unsloth, bartowski, lmstudio-community and AtomicChat: 20 files, enumerated from the tree API rather than sampled. Claims below are bounded to that set.

The quants worth considering on this card:

The complete 4-bit set, so the claim below is reproducible. All 20 four-bit files from the five publishers, ascending. Ten are under 15.6 GiB:

Entries 14 and up exceed the card as weights alone.

Why the 4-bit tier is out, stated as arithmetic rather than as a size floor. Ten of the twenty 4-bit builds are under 15.6 GiB, so "4-bit is too big to load" would be false. The binding constraint is what has to sit beside the weights: projector 0.864 + recurrent state 0.146 + KV at 32K q8_0 1.063 = 2.073 GiB of fixed load. Add that to the smallest uniform 4-bit build, bartowski's IQ4_XS at 14.499, and you get 16.571 GiB — over the card before a single byte of runtime buffer. Shrinking the context does not rescue it: at 16K the same build totals 16.040 GiB, still over; at 8K it totals 15.775 GiB, leaving 0.225 GiB for every compute buffer, the CUDA context, the vision graph and the display driver combined. That is the real reason this recipe is 3-bit, and it holds for all twenty.

Do not read "Q4_K_M" as one number. Across these publishers, files named Q4_K_M alone span 15.656 to 17.671 GiB — a 2.0 GiB spread on one label — and the spread is not noise. ggml-org's convert.log shows their build was quantised with --pure --tensor-type output.weight=q6_k --tensor-type shexp=q8_0 --tensor-type latent=q8_0 --tensor-type attn_=q8_0 --tensor-type ssm_=q8_0, i.e. large parts of it are actually q8_0. A quant name is a recipe the publisher chose, not a size. Always name the publisher and the file.

On quality at this tier, the only comparative measurement found is vendor-published and self-interested: AtomicChat, whose founder posted it and who sell a competing quant set, ran 20 community files plus 16 of their own through one harness on 4× RTX 5090 against their own BF16 logits, and report unsloth's Q3_K_M at 0.0484 mean KLD against their same-size file at 0.0325. Treat the ranking as a vendor claim. The useful, publisher-independent part is their statement that "below 10 GB every quant of this model degrades fast" — which is why this recipe does not send you to a 2-bit build to buy speed. Dropping a tier would buy you a few percent of decode rate for a quality cost the same source calls steep; the bandwidth arithmetic above says the payoff is small, because weights fall faster than throughput rises.

Installation

1. Build or fetch llama.cpp with CUDA

There are three different "which build" claims in circulation for this model, and only one of them is a minimum. Keeping them apart matters, because a reader on a build a few tags old should not conclude they are unsupported.

1. Architecture support floor — old, and any current build clears it. Both structural requirements are already present at b8001, the oldest tag checked: src/llama-arch.cpp carries qwen35 and tools/mtmd/clip-impl.h carries the qwen3vl_merger projector type. Both files returned HTTP 200 at that tag, so this is a real reading and not a 404 misread as absence. b8001 is the floor of the window searched, not the landing build — the true one is older and was not established here. Practically: if your llama.cpp is recent enough to be called recent, it supports this model.
2. A functional floor for a specific capability — none applies to this configuration. This recipe does not depend on any post-release fix that was sourced to a bug report.
3. The build this page was authored against — b10442. That is provenance for the source line numbers and flag names quoted throughout, not a requirement.

What is card-specific here is the CUDA toolkit, and this card asks for less than a Blackwell one. llama.cpp's ggml/src/ggml-cuda/CMakeLists.txt annotates its architecture list directly: compute capability 89 is "RTX 4000, needs CUDA v11.8", and the file appends 89-real to CMAKE_CUDA_ARCHITECTURES as soon as the detected toolkit is 11.8 or newer. The RTX 4060 Ti is an Ada card at sm_89, so any CUDA 11.8+ toolkit produces native code for it — no 12.8 requirement, no 120a-real CMake-version caveat, and a GGML_NATIVE build simply detects the card.

The one genuine build-time requirement in this area is a compile flag, not a version — and it does not apply here either. GGML_CUDA_FA_ALL_QUANTS defaults to OFF in ggml/CMakeLists.txt, and a default CUDA build compiles only four FlashAttention vector instances: f16-f16, q4_0-q4_0, q8_0-q8_0 and bf16-bf16. This recipe's matched q8_0/q8_0 cache is in that default set, so no flag and no newer build is needed. Mixed or q4_1/q5_x cache types are not — which is the mechanism behind the prefill collapse in Troubleshooting.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout b10442   # optional — pins you to the exact tree this page quotes; plain `master` is fine
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j

Video input is compiled in by default — the mtmd CMakeLists.txt sets MTMD_VIDEO ON with the description "enable video support in mtmd (requires ffmpeg binary in PATH)". If you do not have ffmpeg, images still work; only video is affected.

A prebuilt container works too, and is what the ghcr.io/ggml-org/llama.cpp:server-cuda13 image is for. Note it is built without GGML_CUDA_FA_ALL_QUANTS, like any default build.

2. Download the weights and the projector

pip install -U huggingface_hub
hf download unsloth/Qwen3.8-27B-GGUF Qwen3.8-27B-UD-Q3_K_XL.gguf --local-dir ./qwen38
hf download unsloth/Qwen3.8-27B-GGUF mmproj-F16.gguf --local-dir ./qwen38

Both files must come down. Without mmproj-F16.gguf the server loads text-only and rejects images with image input is not supported - hint: if this is unexpected, you may need to provide the mmproj.

Running

./build/bin/llama-server \
  --model ./qwen38/Qwen3.8-27B-UD-Q3_K_XL.gguf \
  --mmproj ./qwen38/mmproj-F16.gguf \
  --alias qwen3.8-27b \
  --host 127.0.0.1 --port 8080 \
  --n-gpu-layers 99 \
  --ctx-size 32768 \
  --parallel 1 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --flash-attn on \
  --batch-size 2048 --ubatch-size 512 \
  --image-min-tokens 1024 \
  --jinja \
  --no-reasoning-preserve \
  --chat-template-kwargs '{"reasoning_effort":"medium"}' \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 \
  --presence-penalty 0.0 --repeat-penalty 1.0

Four of those flags are doing load-bearing work on a 16 GB card and are not defaults:

• --parallel 1. This one is easy to skip and it is worth 0.438 GiB. The common_params struct initialises n_parallel to 1, but that is not the value llama-server runs with: common/arg.cpp overrides it in its per-example defaults block — else if (ex == LLAMA_EXAMPLE_SERVER) { params.n_parallel = -1; // auto by default } — and tools/server/server.cpp then resolves any negative value, logging "n_parallel is set to auto, using n_parallel = 4 and kv_unified = true". You do not have to type anything to get four slots; you have to type --parallel 1 to avoid them. The startup log in issue #27124, from a command line carrying no -np flag at all, prints n_slots = 4, n_ctx_slot = 65536, kv_unified = 'true' — which is the behaviour, observed rather than inferred. Because the recurrent state is allocated per sequence — llama_memory_hybrid gets recurrent_kv_size = std::max((uint32_t) 1, cparams.n_seq_max) — the default costs 627,572,736 bytes instead of 156,893,184: 0.438 GiB, or about 13,500 more tokens of context. Note also that this state is hardcoded GGML_TYPE_F32, so --cache-type-k/v does not shrink it — those flags reach only the 16 attention layers.

  The KV cache itself is not multiplied by the slot count, and it is worth knowing why the two behave differently. The auto path also flips kv_unified to true, and src/llama-context.cpp then sets n_ctx_seq = n_ctx rather than n_ctx / n_seq_max — so -c 32768 allocates 32,768 cells either way. What changes is who owns them: at the default, four slots share one 32,768-token window, so a single long conversation can be evicted by a concurrent request. With --parallel 1 the whole window belongs to one conversation and the recurrent state drops to one copy. On a single-user desktop the flag costs you nothing you wanted.

• --cache-type-k q8_0 --cache-type-v q8_0. Halves KV against fp16. Do not go to 4-bit KV, and do not mix the two types — see Troubleshooting.

• --chat-template-kwargs '{"reasoning_effort":"medium"}' and --no-reasoning-preserve. Thinking is on by default and the model card's default effort is xhigh; the card documents "supported levels are xhigh, medium, and low". Preserved thinking is also on by default and "retains thinking blocks from all historical messages", which on this card means every past reasoning trace keeps occupying your 32K window. On a card that generates at this rate, reasoning effort is also the single biggest lever on how long you wait for an answer — every thinking token is a token generated at the same speed as a visible one.

• --image-min-tokens 1024. llama.cpp itself warns at load time that "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks" — the string is in tools/mtmd/clip.cpp, emitted whenever image_min_pixels is below that threshold. Budget for it: at 1024 tokens and q8_0 KV, each image costs about 35 MB of cache. Image ingestion is prompt processing, which is the half of the workload this card handles comparatively well.

The sampler values are the model card's own thinking-mode recommendation (temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0).

Point any OpenAI-compatible client at http://127.0.0.1:8080/v1. Images go in as image_url content parts; GET /props reports whether the build accepted the projector and whether video is available.

Text-only, if you want the context back

Dropping the projector frees 927,607,488 bytes — about 26,600 more tokens of q8_0 KV. It is a real option on this card and costs you the model's defining capability:

./build/bin/llama-server --model ./qwen38/Qwen3.8-27B-UD-Q3_K_XL.gguf --no-mmproj \
  --n-gpu-layers 99 --ctx-size 65536 --parallel 1 \
  --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on --jinja

Note this buys context, not speed — the weights are unchanged, so the per-token read that sets the decode rate is unchanged too.

Why not Ollama on this card

Ollama's library page for this model lists twelve tags and the smallest is 18 GB — there is no tag below the card's capacity, so any of them will partially offload to system RAM. On a card whose on-GPU bandwidth is already the constraint, pushing layers across PCIe to DDR memory is the worst available trade. Pull the GGUF directly instead.

Results

• Speed: omitted as a measurement, because none exists. There is no benchmark for this model on an RTX 4060 Ti in /check, none in the model card's 100 HuggingFace discussions (all of which were enumerated, and none of which mentions a 4060 Ti at all), and none in llama.cpp's issue tracker. What this page offers instead is set out in What the 288 GB/s bus costs you: a bandwidth ceiling of roughly 20–21 tokens/s that is arithmetic rather than a prediction, and this card's measured position on a different model — 17.9 tok/s on Qwen3 14B at Q4_K and 32K context, against 51.9 for an RTX 5080 at the same capacity. Post real numbers via /contribute and /check/qwen3-8-27b/rtx-4060-ti-16gb stops being empty.
• VRAM usage: derived, not measured — 14.590 GiB of 16 GiB at 32K context with the projector loaded, itemised in the budget table above. Live data, when it exists, will be at /check/qwen3-8-27b/rtx-4060-ti-16gb.
• Quality notes: this card runs the model at 3 bits — see Picking a quant for why 4-bit cannot coexist with the projector — and vision is the part that suffers first. In discussion #69 a community user (a reply in the thread, not its opener) reports that "the multi modal ability is inferior to 3.6 27b when comes to 4 bit quant" — that is a comparison to the previous generation at 4 bits, one tier above what fits here, and no equivalent report exists at 3 bits either way. Expect the text side to hold up better than the vision side, and treat OCR and fine grounding work as things to verify before relying on.

For the full benchmark data, see /check/qwen3-8-27b/rtx-4060-ti-16gb.

Troubleshooting

Prompt processing collapses to tens of tokens per second

You quantised the KV cache to 4 bits, or mixed two cache types. llama.cpp issue #27109, open as of writing, reports prefill dropping from 991–1276 t/s to 34–106 t/s on a qwen35 hybrid model — measured by the reporter on an RTX 3090, and attributed by them to "the CUDA flash-attention / quantized-KV dequant kernel selection for 4-bit KV on this hybrid architecture".

The issue's own title blames q4_1/q4_0, and the reporter's later root-cause comment narrows that considerably — read it rather than the title. GGML_CUDA_FA_ALL_QUANTS is OFF by default, so a default CUDA build compiles only matched K/V FlashAttention instances, and when no instance exists the scheduler silently moves the whole attention op to the CPU backend. By that analysis K=q8_0, V=q8_0 and K=q4_0, V=q4_0 are both "supported, stays on GPU", while K=q4_1, V=q8_0 and K=q4_0, V=q8_0 are not — the mismatch is as fatal as the type. This recipe's matched q8_0/q8_0 is in the supported set. Also from that comment: "Generation speed unchanged" — the failure is prefill-only, so if your generation rate is what disappoints you, this is not your bug and the bandwidth section above is the honest explanation. Rebuilding with -DGGML_CUDA_FA_ALL_QUANTS=ON compiles the rest if you want the mixed types.

It OOMs at startup

The ladder, in the order that costs you least:

1. Drop --ctx-size to 16384. At q8_0 that halves the KV line to 0.531 GiB and buys back 0.531 GiB.
2. Confirm --parallel 1 is actually on the command line — the auto default is 4 and costs 0.438 GiB of recurrent state.
3. Drop the projector with --no-mmproj and run text-only: 0.864 GiB.
4. Drop a quant tier — unsloth's Q3_K_S is 11.711 GiB and UD-IQ3_XXS is 11.095 GiB, but note the quality warning above about going below ~10 GB.

Generation is slower than you expected, and nothing is wrong

Check the ladder above anyway — a partial CPU offload is the one failure that looks like slowness rather than an error. llama-server prints the layer assignment at startup; if any layer landed on the CPU, that is your problem and not the bus. If all 64 are on the GPU, then the bandwidth section is the explanation, and the levers are shorter thinking (reasoning_effort) and shorter context, not a different flag.

Turning on MTP looks like the right fix for a bandwidth-bound card, and probably still isn't

The reasoning is genuinely tempting here, more so than on a faster card: speculative decoding amortises one weight read across several accepted tokens, which is exactly the bottleneck this card has. The model ships a multi-token-prediction head and llama.cpp wires it into speculation — --spec-type draft-mtp works against the inline blk.64 block with no separate draft file. Two things argue against it anyway. It costs VRAM you do not have: enabling it sets n_rs_seq to your draft length, and llama_memory_recurrent allocates mem_size * (1 + n_rs_seq) rows of recurrent state, so --spec-draft-n-max 4 takes the recurrent line from 156,893,184 bytes to 784,465,920 — 0.584 GiB out of 1.410 GiB of headroom. And the only reported measurement is unfavourable: in discussion #80 a user measuring the same weights blob with speculation off and on found "under Ollama, speculation repays its own overhead in the best case and never more", with unpredictable content running 2.3× slower than with no speculation at all. That measurement is on a different runtime and says nothing directly about llama-server, which is precisely why this is worth trying if you have the patience — and worth reporting either way, because on this card it is the most valuable open question on the page.

The model thinks for minutes on a trivial question

Widely reported — one thread is titled "After waiting 49 minutes and 16 seconds while the model was still thinking, I'm simply giving up...". The model card's own control is the lever: reasoning_effort accepts xhigh (the default), medium and low, and thinking can be turned off entirely per request. On a card at the bottom of the bandwidth range this is not a comfort setting — it is the difference between an answer and an evening.

Tool calls fail in agent clients

Multiple reports tie this to the packaged chat template rather than to the quant: llama.cpp issue #27139 is titled "Qwen3.8 Codex error resolved by using the Qwen3.6 chat template file.", and discussion #68 reports the same class of failure with a community-maintained replacement template. If your client's tool calling misbehaves, try --chat-template-file with a corrected template before blaming the 3-bit weights.

Your tool says the file is Q4_K_S

A metadata defect in unsloth's Dynamic builds: Qwen3.8-27B-UD-Q3_K_XL.gguf carries general.file_type = 14, which is Q4_K_S, while the same repo's plain Qwen3.8-27B-Q3_K_M.gguf correctly carries 12. It is a label, not the tensors — the file is 12.518 GiB, which no real Q4_K_S of this model could be. It affects display only.

Vision crashes

One open vision-crash report exists, llama.cpp issue #27124, and it is on a Vulkan build on an AMD Ryzen AI MAX+ 395 under Windows — a different backend and a different vendor from this recipe's CUDA path. No CUDA equivalent surfaced in the llama.cpp issues opened since the model's release. If you hit one on CUDA, that is new information worth filing.