Apodex 1.1 mini on RTX 4060 Ti 16GB: a 36B agent at 128K, where the card is the slower half

What You'll Build

A local, OpenAI-compatible agent endpoint serving Apodex 1.1 mini — the Apache-2.0 agent model Apodex AI fine-tuned from Qwen/Qwen3.5-35B-A3B — on one RTX 4060 Ti 16GB, at a 131,072-token working context, from a Q4_K_M GGUF that is larger than the card.

The weights load 19.908 GiB. The card holds 16. It runs because the model is a sparse Mixture of Experts and llama.cpp will leave the routed experts of the first N blocks in system RAM: 123 tensors carry 18.586 GiB of the file, and across the forty blocks a normal run loads that is 18.164 GiB of the 19.908 GiB resident — 91.2%.

Hardware data: RTX 4060 Ti 16GB (16 GB GDDR6, 128-bit) · 13.546 GiB derived working set at -ncmoe 17, 131,072-token context · See benchmark data

⚠️ Nothing on this page was measured on an RTX 4060 Ti. /check/apodex-1-1-mini/rtx-4060-ti-16gb returns unknown with zero benchmarks. Every figure below is arithmetic over this GGUF's own tensor table and over llama.cpp's allocation code at a pinned release, written out so you can check it. There is no throughput measurement on this page — Results names the space searched and what does exist. If you run it, please send the numbers.

🧭 The conclusion that is specific to this card, before the arithmetic that gets there. Every other offload page in this family ends with your system memory sets the pace, not the GPU. On this card that is false, and the reason is the 128-bit bus. At -ncmoe 17 a decode step reads 0.2601 GB from system RAM and 2.00–3.43 GB from VRAM, depending on how full your context is. The two halves cost the same wall-clock only when your card's bandwidth is 7.7× to 13.2× your DIMMs'. Against the 288 GB/s this catalogue publishes for this board, dual-channel DDR4-3200 gives a ratio of 5.6× and dual-channel DDR5-6000 gives 3.0× — both below the crossover, at every context depth. So on an RTX 4060 Ti the GPU's own memory is between 58% and 81% of the decode roofline, and buying faster DIMMs helps you less here than it would on a card with twice the bus. Where the decode time actually goes prices it out. Everything about capacity on this page is what any 16 GB card gets; everything about speed is not.

ℹ️ This recipe is text-only, deliberately. The checkpoint has a vision tower and the lead GGUF repo ships two projectors for it, but the vendor documents the model as pipeline_tag: text-generation and its model card carries no image-input instructions — the words vision, visual, multimodal and mmproj appear zero times in it (counted 2026-08-30). The smaller projector is also 0.838 GiB, which on this budget is nearly two full steps of -ncmoe spent on a capability the recipe does not use, so the run command refuses it explicitly rather than merely omitting the flag.

⚠️ Two version floors, and the second one is about speed rather than about loading.

• The architecture. LLM_ARCH_QWEN35MOE is absent from src/llama-arch.cpp at b7989 and present at b7990 (it is line 42 at the pin below), so it arrived in the interval (b7989, b7990]. The qwen35moe-specific load_mtp / TENSOR_SKIP wiring that keeps blk.40 off the card arrived in (b10211, b10212]. Both are intervals bracketed by adjacent published tags, which is as fine as a tag-granularity probe resolves.
• A throughput floor on top of that. llama.cpp issue #25162 — "Performance regression on Turing GPUs after 9e58d4d69 — Qwen35 SSM kernels affected", closed as completed 2026-08-03 — records a 24–42% loss in ssm-scan.cu / ssm-conv.cu, the kernels behind thirty of this model's forty layers, with Gemma4 unaffected because it has no SSM layers. Use b10630 or newer — the release the lead quant was produced with — and understand that an older build can cost you silently rather than loudly. ⚠️ That regression was measured on Turing (SM75) and the outcome on Ada is unmeasured. The relevant guard reads # if (defined(GGML_CUDA_USE_PDL) && defined(__CUDA_ARCH__) && __CUDA_ARCH__ >= GGML_CUDA_CC_HOPPER) (ggml/src/ggml-cuda/common.cuh:1650); this card is CUDA capability 8.9, below Hopper, so it compiles the same branch Turing does — which makes the question open, not answered.

Every source line quoted on this page was read at release b10666 (4e97ac86ebe2c4cb8212d98d2641ad6768810896), and line numbers are given for that tree only. The tag moves several times a day — b10701 was published 2026-08-30T18:56:31Z, three tags inside one hour — so this page pins rather than claiming currency. Note also that GET /repos/ggml-org/llama.cpp/releases/latest answers v0.3.0, because every bNNNN tag is marked prerelease; use /releases?per_page=N if you script a version check.

Requirements

Check which card you have before anything else. NVIDIA's own product page gives the RTX 4060 Ti's Standard Memory Config as "16 GB GDDR6 or 8 GB GDDR6" — one name, two capacities, and this recipe needs the larger one. Every row of the ladder below is keyed to a capacity:

nvidia-smi --query-gpu=name,memory.total,clocks.max.memory,pcie.link.gen.max,pcie.link.width.max --format=csv

What the vendor publishes about this board, and what it does not. From that same page: 4352 CUDA cores, 2.54 GHz boost / 2.31 GHz base, 16 GB GDDR6 on a 128-bit interface, Ada Lovelace, CUDA capability 8.9, 165 W, and a Technology Support row "PCI Express Gen 4" whose value is Yes. What it does not publish, counted on the page today: no memory-bandwidth figure, no memory-speed figure, and no PCIe lane count — the strings lanes and x8 occur zero times. Two numbers this page needs are therefore not vendor facts and are labelled as such wherever they appear:

• 288 GB/s of local bandwidth. Not from NVIDIA. It is the spec-table value in Tom's Hardware's RTX 4060 Ti 16GB review and on Hardware Corner's page for the card, and when this page was written (read 2026-08-30) it was the figure our own Qwen3.8-27B page for this card was built on. You can check it yourself without trusting any of us: GDDR is double-data-rate and the bus is 128 bits, so clocks.max.memory × 2 × 16 bytes is your board's bandwidth — 9,000 MHz gives 288 GB/s. ⚠️ That shortcut is specific to GDDR6 and GDDR6X, which carry two bits per cycle; GDDR7 signals PAM3 at 1.5 bits per cycle, so the same arithmetic returns a wrong answer on a 50-series board. This card is Ada with GDDR6, so it holds here and only here.
• A PCIe 4.0 ×8 host link. Also not from NVIDIA. Our MiniMax H3 page for this card documented a ×8 link when this page was written (read 2026-08-30), sourcing it to the tech press and noting there, as here, that the vendor states only the generation. This page asserts no lane count of its own; it tells you where the link would bite and gives you the one command that settles it on your machine. Read pcie.link.width.max from the line above, and see Prefill is the phase the link pays for.

For your DIMMs, do the same arithmetic from their rated transfer rate: dual-channel DDR4-3200 is 3200 MT/s × 8 B × 2 = 51.2 GB/s, dual-channel DDR5-6000 is 96 GB/s, and a single-channel configuration halves whichever you have. Both sides of that comparison matter on this page, and here — unusually — the card's side matters more.

Installation

1. Build llama.cpp with CUDA

The architecture string in this GGUF is qwen35moe, registered as LLM_ARCH_QWEN35MOE in src/llama-arch.cpp. AD106 is compute capability 8.9; the default CUDA build covers it and no architecture flag is needed.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout b10666
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j

2. Download the Q4_K_M GGUF

pip install -U huggingface_hub
hf download bartowski/apodex_Apodex-1.1-mini-GGUF \
    apodex_Apodex-1.1-mini-Q4_K_M.gguf --local-dir ./apodex-1.1-mini

bartowski/apodex_Apodex-1.1-mini-GGUF apodex_Apodex-1.1-mini-Q4_K_M.gguf is 21,864,082,336 bytes (20.363 GiB), imatrix-calibrated, produced on llama.cpp b10630. Repo lastModified 2026-08-26T09:14:35Z; the file list, the byte count and the download URL were re-fetched and HEAD-checked on 2026-08-30 rather than copied from a sibling page.

Disk size and VRAM residency are different numbers here, and the gap has two parts. The file is 21,864,082,336 bytes (20.363 GiB) on disk; a normal run loads 21,376,133,632 (19.908 GiB). The difference is 487,948,704 bytes, and only most of it is the model. 476,956,672 of it is blk.40, the multi-token-prediction head, which is created with TENSOR_SKIP unless you ask for it (src/models/qwen35moe.cpp). The remaining 10,992,032 bytes (10.5 MiB) is the file's non-tensor content — the GGUF header, the key-value metadata, the tensor-info table and the alignment padding — and it is exactly 21,864,082,336 − 21,853,090,304, the file size minus the sum of its tensor payloads. So the MTP block alone does not account for the gap. Budget 22 GB of disk and 19.908 GiB of weights, and do not substitute one for the other.

Do not start from the publisher's quickstart on this card. That line is llama-server -hf bartowski/apodex_Apodex-1.1-mini-GGUF:Q4_K_M, and the card itself says "llama.cpp downloads the mmproj automatically when using" -hf. It does: find_best_mmproj matches any sibling filename containing mmproj, this repo ships two (899,283,488 and 902,822,432 bytes), and no_mmproj defaults to false while mmproj_use_gpu defaults to true (common/common.h:594,596). So the quickstart puts 0.838 GiB of vision projector on a card this recipe has already budgeted to within about a third of a gigabyte. Download the file explicitly, as above, and load it with -m.

Q4_K_M on a 16 GB card is a choice, and the alternative is narrower than it looks. Ten of bartowski's rungs are under 16 GiB on disk, so "small enough to load" is not the constraint — what has to sit beside the weights is. Hold this page's own ceiling (derived below: 13.917 GiB for weights plus KV plus recurrent state) and a 32,768-token q8_0 cache, and a fully resident configuration needs weights at or below 13.524 GiB. The whole 3-bit tier misses that: IQ3_XXS is 14.287 GiB and Q3_K_S is 14.886. The escape from offload is not "drop to 3-bit", it is "drop to 2-bit" — Q2_K_L at 12.650 GiB, or IQ2_M at 11.682. That is exactly where a second publisher draws the line: abenzerps's hardware table gives the 16 GB row as "IQ2_M full offload; IQ3_M may require hybrid offload" and then closes the section with "Prefer Q4_K_M or higher when system memory permits." This recipe follows that closing sentence, because the model is an agent that runs tool calls and multi-step plans, and a malformed argument is a failed task rather than a clumsy sentence.

Running

./build/bin/llama-server \
    -m ./apodex-1.1-mini/apodex_Apodex-1.1-mini-Q4_K_M.gguf \
    --no-mmproj \
    -ngl 99 \
    -ncmoe 17 \
    -np 1 \
    -c 131072 \
    -ctk q8_0 -ctv q8_0 \
    -fa on \
    --load-mode none \
    -cram 2048 \
    --temp 1.0 --top-p 0.95 --repeat-penalty 1.05 \
    --host 127.0.0.1 --port 8080

Then talk to it:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "apodex-1.1-mini",
    "messages": [{"role": "user", "content": "Plan a three-step verification pass for a CSV of quarterly revenue."}]
  }'

Nine of those flags are decisions rather than defaults. Each default below was read at b10666 for this page rather than carried across from another card, and each decision is labelled capacity-driven — meaning it is the same on any 16 GB card — or bandwidth-driven, meaning it is not.

• --no-mmproj refuses the vision projector rather than merely not asking for it: 0.838 GiB, for the reason under Installation. Capacity-driven.
• -ngl 99 puts every layer on the GPU as the starting point; -ncmoe then walks part of it back off. Capacity-driven — it is the baseline the offload is measured back from, and it is also what keeps the KV cache and the recurrent state on the card.
• -ncmoe 17 keeps the routed experts of blocks 0–16 in system RAM. The flag's help text at this pin is "keep the Mixture of Experts (MoE) weights of the first N layers in the CPU" (common/arg.cpp:2790) and the implementation matches the wording exactly. 17 is derived in Choosing your own -ncmoe as the lowest rung that clears this page's ceiling at 131,072 tokens — it is not a measured optimum, and it is not inherited. Capacity-driven.
• -np 1. llama-server sets params.n_parallel = -1; // auto by default (common/arg.cpp:1400) and resolves that to four slots sharing one KV pool. The Gated-DeltaNet recurrent state is allocated per sequence, so four slots take it from 0.061 GiB to 0.245 and split your context four ways. One slot is what a single-user agent loop wants. Capacity-driven.
• -c 131072 is half the model's declared context_length of 262144. The ladder below prices the other half; it costs three more steps of -ncmoe. Both, and it is this page's largest speed lever — capacity-driven for the 1.328 GiB the cache occupies, and bandwidth-driven because that cache is re-read from VRAM on every generated token. Context depth moves the crossover from 7.70× on an empty window to 13.18× on a full one, which is a larger throughput decision than the offload depth is.
• -ctk q8_0 -ctv q8_0 quantises the KV cache. block_q8_0 is a 2-byte scale plus 32 one-byte quants — 34 bytes per 32 elements, i.e. 8.5 bits per element, not 8. It takes the 131,072-token cache from 2.500 GiB to 1.328, which on this card is a saving of 1.172 GiB — worth between two and three steps of -ncmoe you do not have to spend, since a step costs 0.4219 or 0.4863 GiB so two come to 0.844–0.973 and three to 1.266–1.459. Capacity-driven — and, on this card, it buys speed too: the KV cache is read out of VRAM on every decode step, so halving it halves the largest single term in the decode budget.
• -fa on makes Flash Attention explicit. The default at this pin is auto — flash_attn_type = LLAMA_FLASH_ATTN_TYPE_AUTO (common/common.h:499) — and auto would turn it on anyway here, because a quantised V cache requires it. Pinning it means a future change to what auto decides cannot silently change your memory profile. Capacity-driven.
• --load-mode none. When you override tensors to CPU while memory-mapping the file, the loader warns: "tensor overrides to CPU are used with mmap enabled - consider using --load-mode none for better performance" (src/llama-model-loader.cpp:1194). Taking that advice makes the 7.752 GiB of host-side experts ordinary allocations rather than file-backed pages, which is also why the RAM line in Requirements is a hard figure and not a cache you can squeeze. Bandwidth-driven, on the host side: the point of the pinned host buffer is the rate the CPU reads those experts at during decode.
• -cram 2048 caps llama.cpp's host-memory prompt cache at 2 GiB. The default is 8192 MiB — cache_ram_mib = 8192 at common/common.h:632 — and it costs more on this architecture than on a plain transformer. See System RAM under Troubleshooting. Capacity-driven, but on host RAM rather than on VRAM — it does not appear anywhere in the card's budget.

Sampling values are the vendor's own from the model card (temperature: 1.0, top_p: 0.95, repetition_penalty: 1.05), whose text notes that Apodex "follows the Qwen3.5 chat template"; llama-server uses the template embedded in the GGUF. Pass tool schemas through the API's tools= field rather than inlining them in the system prompt — that is the vendor's explicit instruction.

How the offload actually works

-ncmoe N is not an architecture feature and does not know what a Mixture of Experts is. It appends one buffer-type override per block index 0 … N-1, each built from a regex in common/common.h:1130:

const char * const LLM_FFN_EXPS_REGEX = "\\.ffn_(up|down|gate|gate_up)_(ch|)exps";
// llm_add_n_cpu_ffn_overrides(N, LLM_FFN_EXPS_REGEX, params.tensor_buft_overrides)

The loader matches that against tensor names and, on a hit, gives the tensor a CPU buffer type at tensor-creation time, before any buffer exists — so those bytes are allocated in host memory and never enter VRAM at all. Nothing in the path consults the architecture, which is why it works identically on this hybrid recurrent model and on a plain transformer MoE. 123 tensors in this file match, three per block across all 41 blocks, totalling 18.586 GiB, of which 18.164 GiB sit in the forty blocks a normal run loads.

Three things it does not move, all of which stay on the card and all of which show up in the decode budget later:

• The attention and Gated-DeltaNet weights. A block's attn_* tensors and its ssm_conv1d / ssm_a / ssm_alpha / ssm_beta / ssm_norm / ssm_out do not match _exps.
• The shared expert and the router. ffn_*_shexp and ffn_gate_inp do not match either, so the dense path every token takes stays resident, and routing is decided on the GPU.
• The KV cache and the recurrent state. Both take their buffer type from the layer's device, which -ngl alone decides; -ncmoe rewrites weight tensors and never touches the layer assignment. A block whose experts are in RAM still keeps its recurrent state in VRAM — which is what makes the budget below decomposable at all.

Add it up and offloading everything the regex can reach, -ncmoe 40, still leaves 1.744 GiB resident: 0.655 GiB of token_embd and output, and 1.089 GiB of everything else across forty blocks.

Does that actually happen on this architecture? The source says so, and there is an independent report that the offload takes effect on a hybrid MoE: llama.cpp issue #27698 records a run with --fit off --n-gpu-layers 46 --n-cpu-moe 6 where "the weights load (expert offload is honored)" and the failure comes later. ⚠️ That report is an RX 6700 XT (gfx1031, RDNA2) on ROCm and its offload-honoured branch is a gemma4moe model rather than this one, so nothing about the failure transfers to this card — it is cited for the one clause quoted, plus a magnitude in the next section.

The VRAM budget

Weights, KV cache and recurrent state at the recipe's settings. Every figure is derived from the GGUF's own tensor table and from llama.cpp's allocation code at b10666.

The three rows come from:

• Weights. The file's tensor payloads sum to 21,853,090,304 bytes, of which 476,956,672 are blk.40, leaving 21,376,133,632 (19.908 GiB) resident. -ncmoe 17 moves 8,323,596,288 of that (7.752 GiB) to the host.
• KV cache. block_count is 41 and nextn_predict_layers is 1, so forty blocks load; full_attention_interval 4 makes ten of them full-attention — blocks 3, 7, 11, 15, 19, 23, 27, 31, 35, 39. Each carries head_count_kv 2 at key_length and value_length 256, so 1,024 cache elements per token per layer and 10,240 across the ten. At q8_0 that is 10,880 bytes per token; at f16 it is 20,480. The other thirty blocks are Gated-DeltaNet and cache nothing that grows with context.
• Recurrent state. Those thirty blocks each allocate n_embd_r() = (ssm_d_conv − 1) × (ssm_d_inner + 2 × ssm_n_group × ssm_d_state) = 3 × (4096 + 2×16×128) = 24,576 elements and n_embd_s() = ssm_d_state × ssm_d_inner = 128 × 4096 = 524,288 elements, both F32 (src/llama-hparams.cpp:183,211). That is 2,195,456 bytes per layer per sequence and 65,863,680 across thirty. llama-server prints it at startup as RS buffer size.

The two terms that table cannot produce, and what this page reserves for them

13.546 GiB is not a peak. Two things sit outside it, and on a card this tight they decide whether a configuration loads. Rather than leave them as a warning, this page reserves for them explicitly.

1. A 16 GB card does not give you 16 GiB. The CUDA context, the driver and anything driving a display take a share the model budget never sees. The only expert-offload run in our catalogue with the numbers written down is gemma4-26b on an RTX 3060 at -ncmoe 12, whose submitter recorded a peak of 11,179 MiB of that card's 12,288 on a headless box — so at most 1,109 MiB was unavailable to the model there. ⚠️ That is a different board in a different VRAM tier, and the term is not proportional to capacity (it is a context plus a driver reservation, not a percentage), so borrowing the absolute 1,109 MiB onto a 16 GB card is a transfer of an absolute quantity and not a measurement of this one. It is the best number in the catalogue and it is the direction this page errs in: this page reserves 1,109 MiB, and a Windows box driving a monitor should expect to want more.

2. The compute buffers. llama.cpp allocates graph working memory on top of everything above, and under expert offload it also stages host-resident expert tensors on the GPU during prefill — each of this file's expert triples is 0.4219 or 0.4863 GiB, so that staging term alone is of that order. This page reserves 1024 MiB, which is llama.cpp's own per-device auto-fit margin: fit_params_target is initialised to 1024 * 1024*1024 at common/common.h:481. It is the runtime's opinion of how much room it wants, not a figure this page invented. For scale on leaving nothing: #27698 above records a clean weight load followed by context creation failing on a ~504 MiB compute buffer with cudaMalloc failed: out of memory — a different card, a different backend and a different model, quoted only for the magnitude of a term every table in this family excludes and for the shape of the failure it produces.

So the rule this page sizes to is:

weights + KV + recurrent state ≤ 16,384 − 1,109 − 1,024 MiB = 14,251 MiB = 13.917 GiB.

At -ncmoe 17 and 131,072 tokens the sub-total is 13.546 GiB, which clears that by 0.371 GiB. Read your own numbers off the server's startup log rather than trusting mine: llama.cpp prints model buffer size once per buffer, so under -ncmoe you get a CUDA line and a host line and the split above is directly checkable, followed by KV buffer size, RS buffer size and compute buffer size. If you do, those figures are the most useful thing you could send us — more useful than a speed number, because they are the two terms nobody in this family of recipes has been able to derive.

Choosing your own -ncmoe

Every row is weights + KV + recurrent state, q8_0 KV, one slot. Free of 16 GiB is what the naive arithmetic says; Slack is what is left once the driver share and the compute reserve are taken out, and the two columns disagree by more than 2 GiB, which is the whole point of the previous section.

Two rows are worth reading twice. -ncmoe 16 at 131,072 tokens has 1.968 GiB "free" and does not load — that is what the ceiling is for. And -ncmoe 17 is the recipe's setting because it is the lowest rung that clears the ceiling at this context, which is a derivation and not a preference: 16 misses by 0.115 GiB. Going deeper is always safe and always slower; the bottom row is the model's entire declared 262,144-token window, which this card can hold at -ncmoe 20.

Then stop guessing and measure it. llama.cpp ships a tool that answers this on your hardware without running the model:

# what the fitter would choose on your box, as CLI arguments
./build/bin/llama-fit-params -m ./apodex-1.1-mini/apodex_Apodex-1.1-mini-Q4_K_M.gguf -c 131072

# estimated MiB per device: model, context, compute
./build/bin/llama-fit-params -m ./apodex-1.1-mini/apodex_Apodex-1.1-mini-Q4_K_M.gguf -c 131072 -fitp on

And when llama-server exits it prints llama_memory_breakdown_print, a per-device split of total / free / model / context / compute / unaccounted. That compute column is what this page reserved 1024 MiB for, and unaccounted is your driver's share.

Why the steps are uneven

A step of -ncmoe does not cost a fixed amount of VRAM in this file, and any advice of the form "one step buys X GB" is wrong here. bartowski's recipe upcasts ffn_down_exps to Q6_K in exactly twenty of the forty loaded blocks, so a block's three expert tensors are either 452,984,832 bytes (0.4219 GiB, all Q4_K) or 522,190,848 (0.4863 GiB, Q6_K down-projection). The expensive twenty are blocks 0–4, then 7, 10, 13, 16, 19, 22, 25, 28, 31, then 34–39 — the top and bottom of the stack are all expensive and the middle alternates. Read the cumulative column above rather than multiplying. Nine of the seventeen blocks this recipe offloads are the expensive kind, which is why -ncmoe 17 moves 7.752 GiB rather than seventeen equal shares of anything.

This is a property of the file, not of the model. IQ4_XS from the same publisher is 17.955 GiB on disk and its ladder is linear. If you would rather spend quality than offload depth, that is the trade — but derive its table from its own tensor list; do not scale this one.

Where the decode time actually goes

This is the section that is not the same on a faster 16 GB card, and it is the reason the capacity numbers above are interchangeable across the tier while the performance advice is not.

The mechanism first. An op whose weights live in a host buffer runs on the CPU — "operations with weights are preferably run on the same backend as the weights" (ggml/src/ggml-backend.cpp:942) — unless the scheduler finds "a backend with higher prio wants to offload the op" (:960). CUDA answers that with a single comparison:

// ggml/src/ggml-cuda/ggml-cuda.cu:5341-5344 at b10666
static bool ggml_backend_cuda_device_offload_op(ggml_backend_dev_t dev, const ggml_tensor * op) {
    ...
    return get_op_batch_size(op) >= dev_ctx->op_offload_min_batch_size;
}
// :5515 — const int min_batch_size = getenv("GGML_OP_OFFLOAD_MIN_BATCH") ? atoi(getenv(...)) : 32;

Generating a token is a batch of one, and one is less than 32, so at decode the offloaded expert matmuls stay on the CPU and those weights never cross PCIe at all. Prompt processing is a batch of hundreds, so at prefill those same tensors are copied host→device. The two phases are bound by different things and this page treats them separately.

The decode roofline, in bytes

expert_used_count is 8 of expert_count 256, so exactly one thirty-second of each block's expert weight is read per token. Everything else on the card is read whole. At -ncmoe 17:

Two things there are new, and they are why this page's conclusion differs from its siblings'.

The KV cache is decode traffic, not just decode capacity. Every generated token reads the whole occupied cache back out of VRAM. At a full 131,072-token window that is 1,426,063,360 bytes — the single largest term in the table, larger than every weight the card holds put together. It is easy to price the KV cache as a capacity line and then leave it out of the bandwidth arithmetic — this page's own budget table above treats it as capacity, and so does every offload budget in this family — and on a card where VRAM bandwidth is the scarce resource, leaving it out changes the answer rather than merely rounding it.

The output.weight head is separated from the embedding table. They are usually lumped as "0.655 GiB of embeddings", but only one of them is read per token: output.weight is 417,177,600 bytes of Q6_K read in full to produce logits over 248,320 tokens, while token_embd.weight is read one row at a time — 1,152 bytes. Splitting them takes the non-expert VRAM read from a loose upper bound of 1.744 GiB down to a near-exact 1.4776 GiB, and because the bound was pointing the wrong way for the conclusion, tightening it makes the conclusion stronger.

The crossover, and where this card sits

The two halves of a decode step take equal wall-clock exactly when your card's bandwidth is VRAM bytes ÷ host bytes times your DIMMs'. That ratio needs no bandwidth figure at all, which is why it is the form this page states:

Now put your two bandwidths against it. Using the 288 GB/s this catalogue publishes for this board — a tech-press figure, not a vendor one, and checkable from your own clocks.max.memory as described under Requirements:

On dual-channel DDR4-3200 or faster, at any context depth, this card's own memory is the larger half of a decode step — and on slower or single-channel memory it becomes the larger half as the context fills: by 32,768 tokens on dual-channel DDR4-2133, and by this recipe's window on every row of the table. That inverts the advice the rest of this family gives, and the inversion has two independent causes that push the same way: this recipe offloads seventeen blocks where a 12 GB card offloads twenty-six, so the host share is smaller; and this board reads its own memory over 128 bits where its tier-mates use 256. Which terms a crossover contains decides its value, and two numbers are in circulation that are not the same calculation. The figure usually quoted for this technique is 5.3×, from a 12 GB card at -ncmoe 26: it divides a loose ≤ 1.744 GiB bound on non-expert VRAM by the host read, and it excludes both the KV cache and the recurrent state. Every crossover on this page includes the recurrent state, and the three rows above add the KV cache at the stated occupancy. Run that same 12 GB case on this page's tightened bound and the two questions answer 4.74× (recurrent state in, KV out) and 6.55× (both in); the loose bound with both in gives 7.3×. A 384-bit board passes all four comfortably, which is the point — that conclusion is sound at that depth on that capacity. Two things move it here and they push the same way: seventeen offloaded blocks instead of twenty-six, and a KV term this page counts. So the conclusion flips on the bus, not on the arithmetic alone.

Priced out as a roofline — the two halves treated as serial, which they broadly are, since within a layer the CPU's expert matmul consumes what the GPU's router produced:

⚠️ Those are ceilings from two bandwidth terms, not predictions. They exclude the CPU time to actually multiply the experts, kernel launch overhead, attention arithmetic, sampling and everything else, and they credit both memories with perfect efficiency. Read them as a ceiling you cannot beat, never as a rate you will reach. On an empty context the same arithmetic gives 58 / 68 / 83 / 104 tok/s down the same four rows, with the card's share falling to 41–72%.

And the practical consequence is the opposite of the usual one. Moving from DDR4-3200 to DDR5-6000 improves this card's 131K roofline by 1.16×. On a card with roughly three times the bandwidth the same upgrade is worth about 1.36×, because there the host half is a larger fraction of a smaller total. Faster DIMMs are worth less on this card than on a fast one — which is the same statement as "this card is the bottleneck", said from the other end. If you are choosing where to spend, note that the one thing that helps the VRAM half at this recipe's settings and costs nothing is already in the run command: -ctk q8_0 -ctv q8_0 removes 1.328 GiB per token at a full window, roughly 4.95 ms of the 11.90.

The offload depth is still the lever, and it is priced per step. Moving one block from card to host frees its whole expert triple from VRAM — 0.4219 or 0.4863 GiB — and adds one thirty-second of it to every token's host read: 13.5 MiB for a cheap block, 15.6 MiB for an expensive one. On dual-channel DDR4-3200 that is 0.28–0.32 ms per token added to the host side, against 0.05 ms saved on the card side, so each step of -ncmoe costs roughly a quarter of a millisecond per token. Going from this recipe's 17 to 20 for the full 262,144-token window costs about 0.8 ms per token in offload — and adds 1.328 GiB of KV traffic, about 4.95 ms, which is six times more. At this recipe's depth, context is a far more expensive throughput decision than offload depth is.

Prefill is the phase the link pays for

At batch sizes of 32 or more the scheduler hands the expert matmuls to the GPU, which means copying host-resident expert tensors across PCIe. llama.cpp issue #25859 is an open profiling report against exactly this shape of run — "My box: RTX 3060 12GB, Ryzen 5600X, 32GB DDR4, PCIe 4.0", "Qwen3.6-35B-A3B (Q4_K_M), -ncmoe 26" — and it finds "prefill is bottlenecked by PCIe transfers, not GPU compute", with the GPU idle around 42% of each pass on expert H2D copies that run serially with the matmuls consuming them. ⚠️ Carry its caveats: a different model, one box, gen-4-specific, and the author dates the measurement to "around the b6xxx era", so the idle fraction may be stale. It is the strongest evidence in circulation that the shape of this path is real, and it is not a measurement of Apodex.

If the whole offloaded set crosses per physical batch — the shape that report describes — then at -ncmoe 17 that is 8.32 GB per -ub batch, and -ub defaults to 512 (common/common.h:452; -b defaults to 2048 at :451). PCIe 4.0 carries about 1.97 GB/s per lane after encoding, so the transfer term alone bounds prompt processing at roughly 1,940 tokens/s over sixteen lanes and 970 over eight — theoretical link rates, with real achieved throughput lower.

This is the one place a lane count changes the answer, and this page does not assert yours. NVIDIA publishes the generation and not the width; our MiniMax H3 page for this card documented a PCIe 4.0 ×8 link on the strength of a tech-press review when this page was written (read 2026-08-30), and this page neither repeats that as a vendor fact nor contradicts it. Read your own:

nvidia-smi --query-gpu=pcie.link.gen.max,pcie.link.width.max,pcie.link.gen.current,pcie.link.width.current --format=csv

If width.max reads 8 where a tier-mate reads 16, halve the ceiling above; if gen.current reads 3 where gen.max reads 4, you are in a slot that halves it again, silently. Either way it is a time-to-first-token problem and not a tokens-per-second one, because at decode batch 1 those weights do not move. And if the link turns out to be your constraint, --no-op-offload disables the offload rule globally, keeping the expert matmuls on the CPU in both phases and removing the transfers entirely — a real trade, untested here, and one flag to try.

Does the arithmetic survive contact with a real card?

Nobody has run Apodex on an RTX 4060 Ti, so the honest answer is that this page is derived and the derivation has three partial checks, none of them on this pair.

The offload ladder has a measured analogue one tier down. gemma4-26b on an RTX 3060 is the only expert-offload run in our catalogue with a written-down ladder: -ncmoe 12 gave 37.23 tok/s at an 11,179 MiB peak and -ncmoe 30 gave 21.20 tok/s on the same unchanged card — eighteen more blocks on the host path costing 43% of the generation rate — with -ncmoe 8 and below failing to load outright. Different model, different card, DDR4-2133 dual channel, headless, llama.cpp b10088. The direction and the magnitude of the effect are the point; none of those numbers is yours.

The prefill claim has the profiling report above, with the caveats stated there.

And the 131,072-token window has a control run on this model's base architecture. llama.cpp issue #27756 is a long-context failure report against Qwen3.8-27B — a different 64-layer member of this family, 48 of whose layers are Gated-DeltaNet — which emits EOS as its first generated token beyond roughly 98–130k positions on CUDA, on CPU, and on a fork. Its reporter ran a cross-model control on the same build, same server and same machine: "The same needle test at 243k tokens PASSES on Qwen3.5-35B-A3B (30 GDN layers)" — which is exactly this model's base and exactly this model's thirty Gated-DeltaNet layers. ⚠️ It is one box, one control run reported in passing, a needle-recall test rather than a quality evaluation, and the base model rather than this fine-tune. It is not a guarantee. It is the only evidence found in the space searched (the llama.cpp tracker and the 22 model cards enumerated under Results) that this architecture's recurrent state holds up past the window this page documents, and it points the right way.

Results

• Speed: omitted, and the reason is a predicate rather than a count. No throughput figure for this GGUF, under llama.cpp, on any NVIDIA card, at any offload depth exists in the space searched on 2026-08-30. That space was: the model cards of all 22 HuggingFace repositories matching Apodex-1.1-mini across 11 namespaces, re-enumerated today via GET /api/models?search=Apodex-1.1-mini&limit=100 and fetched one by one (the count was 21/10 two days ago, so re-run it rather than trusting this sentence; one of the 22, aykutx21/Apodex-1.1-mini-GGUF, holds nothing but a .gitattributes and has no card at all); the llama.cpp issue tracker, where repo:ggml-org/llama.cpp apodex returns 0 against 271 for qwen35moe and 247 for n-cpu-moe on the same query shape; and our own /check, which returns unknown with zero benchmarks. Figures for the model do exist, and every one is a different artifact on a different engine:

  • kingjones777 publishes four of his own repacks at 64.87 / 63.30 / 45.05 / 32.54 tok/s, in ROCmFP4/ROCmFPX tensor formats only a fork of llama.cpp opens, on a Ryzen AI MAX+ 395 (gfx1151) unified-memory APU at full offload — a machine with no expert offload and no PCIe hop, which is the entire subject of this page. Read the top of that range as unsettled: that build's own card states the run is still pending on an idle box while the shared variants table prints 64.87 for it, under a caption reading "A dash means I haven't measured that one yet".
  • ghazni101 reports "generation measured at 226 tok/s end-to-end for MQ4R". Three disqualifications, all verified: it is a .mq4r file under hipfire, a Rust engine that is not llama.cpp; .mq4r is not a GGUF; and it is an RX 7900 XTX, i.e. gfx1100 — name the target, because one gfx id away is a different claim.

  One measurement on this pair would be worth more than all of the above. If you run it, contribute it.

• VRAM usage: 13.546 GiB derived sub-total at -ncmoe 17 and 131,072 tokens, plus the ~1,109 MiB driver share and 1024 MiB compute reserve this page sizes against but cannot derive. 7.752 GiB of expert weights sit in system RAM alongside. See /check/apodex-1-1-mini/rtx-4060-ti-16gb for live data as it lands.

• What to expect anyway, as a mechanism rather than a number: decode reads 0.26 GB from your DIMMs and 2.0–3.4 GB from the card per token, so on a dual-channel box this card — not your memory — sets the pace, and the gap between it and a faster 16 GB board is smaller than their spec sheets imply but real. Prefill is where the host link is on the critical path, and it is the only phase where a narrow slot costs you anything.

• Quality notes: Q4_K_M on a model whose routed experts are already narrow (expert_feed_forward_length 512), which is part of why the sub-4-bit rungs give up more than their file sizes suggest. The lead quant is imatrix-calibrated and the publisher ships both the calibration corpus and the imatrix beside the ladder, which is the reason to prefer it over an uncalibrated conversion at the same nominal tier. The vendor publishes agentic evaluation scores on the model card, but they are run in Apodex AI's own harness and several of the benchmarks are the vendor's own creations, so they are a vendor claim and are not reproduced here.

For the full benchmark data, see /check/apodex-1-1-mini/rtx-4060-ti-16gb.

Troubleshooting

unknown model architecture: 'qwen35moe'

Your binary predates LLM_ARCH_QWEN35MOE, which arrived in (b7989, b7990] — so this means a genuinely old build, not a merely recent one. Check llama-server --version against the release list and move to b10630 or newer.

It loads, then runs out of memory when a long prompt lands

Two causes with opposite fixes.

The KV cache grows at 10,880 bytes per token, so a configuration that loads cleanly can still die deep into a context. Check your -c against the ladder and either raise -ncmoe or lower -c. On this file, raising -ncmoe by two frees between 0.844 and 0.973 GiB depending on which blocks the steps land on.

The other cause is specific to expert offload. During prompt processing — any batch of 32 or more tokens on CUDA — the scheduler moves the host-resident expert matmuls onto the GPU, which means staging those tensors there. Each of this file's expert triples is 0.4219–0.4863 GiB, so prefill peaks materially above the generation footprint the budget table describes. If you OOM while a long prompt is being processed but not while generating, try --no-op-offload, which keeps those ops on the CPU where their weights already are; expect slower prefill in exchange. If the card is also driving a display, start one or two steps deeper than the table and read nvidia-smi after load.

System thrashing, swapping, or an OOM kill with VRAM to spare

Your system RAM is the limit, and there are two terms in it.

The first is the offload: at -ncmoe 17 this recipe puts 7.752 GiB of expert weights on the host, and with --load-mode none those are real allocations rather than file-backed pages.

The second catches people, because nothing in the command asks for it. llama-server keeps a host-memory prompt cache, on by default at 8192 MiB — cache_ram_mib = 8192 at common/common.h:632, exposed as -cram / --cache-ram at :1713 with -1 for no limit and 0 to disable, added by PR #16391 whose body says the cache "is stored in regular RAM". So a default server may hold up to 8 GiB beyond the offload — close to 16 GiB of host memory before the operating system gets any, which is why the command above pins it to 2.

There is one direct measurement of that growth on this architecture. llama.cpp #27894 was opened as "Hybrid SSM/attention models (qwen35moe) leak a fixed ~126 MiB RSS per request" — 40 requests taking RSS from 1340 to 6379 MiB, perfectly linear — and then retracted by its own author once a contributor pointed at --cache-ram: the decisive run is --cache-ram 512 with nothing else changed, where RSS climbs by the same ~126 MiB per request until it reaches +507 MiB against the bound and then stays flat for ten consecutive requests. The growth is the prompt cache, it is bounded by the flag, and the flag is the fix. Closed as completed, 2026-08-28. Two cautions if you cite it: the 126 MiB is the only measured per-entry figure anywhere and it is roughly twice one of this model's recurrent states (65,863,680 bytes = 62.8 MiB), so the reporter's identification of an entry as one state is an inference the arithmetic does not support even though the 126 stands as a measurement; and it was measured on a different model of the same architecture, and rebuilt with -DGGML_VULKAN=ON gave +126.0 MiB per request against ROCm's +126.3 — so this is a model-and-request-layer behaviour and not a vendor story.

So 32 GB is the practical floor for this recipe as written and 16 GB is not enough. On a 32 GB box keep -cram 2048, or pass -cram 0 to disable caching entirely and trade prompt-reuse speed for headroom.

Generation is slower than you expected

Check three things, and note that only one of them is the GPU.

Your DIMMs, first, but expect less from them here than elsewhere. Single-channel memory halves the host half of every decode step — 5.08 ms becomes 10.16 on DDR4-3200 — which on this card is a 1.3× loss rather than the near-2× it would be on a faster board. Confirm dual channel before anything else; it is the cheapest fix and it is not the largest one on this recipe.

Your context depth, second, and it is the largest. At a full 131,072-token window the KV cache is 1.328 GiB of VRAM traffic per generated token, larger than every weight on the card put together. Generation slowing down as a conversation grows is not degradation, it is the roofline. If your agent loop never fills a six-figure window, -c 65536 at -ncmoe 16 is a row in the ladder above and halves that term.

Your offload depth, third and smallest. Each step is 13.5–15.6 MiB more read from RAM per token, about a quarter of a millisecond on dual-channel DDR4-3200. Do not go deeper than the ladder requires, but do not expect much back from going shallower either.

If it is specifically the wait before the first token that is bad while generation is fine, that is the bus and not the model — see Prefill is the phase the link pays for. And if you have GGML_CUDA_DISABLE_GRAPHS set anywhere in your environment, unset it rather than setting it to 0: ggml/src/ggml-cuda/common.cuh:1258 reads getenv("GGML_CUDA_DISABLE_GRAPHS") != nullptr — presence, not value — so =0 switches CUDA graphs off exactly as thoroughly as =1 does.

If you leave -ncmoe off entirely

llama.cpp will pick an offload for you. Auto-fit is on by default — fit_params = true at common/common.h:476 — and moves MoE tensors to system memory until it can leave its 1024 MiB per-device margin. That is a reasonable path with two caveats: it records its choice nowhere except the log, so you cannot reproduce a run from the command line alone; and there is an open regression report against the margin flag on this architecture family — llama.cpp issue #27171, filed 2026-08-16 against Qwen3.6-35B-A3B Q4_K_M with --fit-target, still open and labelled bug-unconfirmed as of 2026-08-30, with a proposed fix (PR #27207) still open and unmerged.

You do not need -fit off alongside an explicit -ncmoe. The fitter refuses to run when you have set your own overrides — common/fit.cpp:484 throws model_params::tensor_buft_overrides already set by user, abort, which is caught and logged as a warning while the run continues with your settings. Seeing that line means your offload is being honoured, not that something broke.

Should you turn on the MTP head?

The GGUF carries a working multi-token-prediction block and llama.cpp drives it with --spec-type draft-mtp — the exact spelling matters. On this card, price it first, because every cost lands on the 0.371 GiB of slack this page reserved:

• blk.40 stops being skipped: +0.444 GiB, and -ncmoe 17 does not touch it — the flag installs overrides for blk.0 through blk.16 only.
• The recurrent state is widened by the draft depth — and by one group more than the flag names. src/llama-memory-recurrent.cpp:101 sizes the cache as mem_size * (1 + n_rs_seq), and n_rs_seq is what need_n_rs_seq() at :394 returns, namely draft.n_max. So --spec-draft-n-max 3 allocates four groups rather than three: 0.245 GiB in total, an increment of 0.184 GiB over the single-slot 0.061.
• A second, single-layer KV cache is allocated for the draft context, f16 by default and untouched by your -ctk — it has its own flag, --spec-draft-type-k — so at 131,072 tokens on one layer that is 2,048 bytes per token, 0.250 GiB.

That is 0.444 + 0.184 + 0.250 = 0.878 GiB against 0.371 GiB of slack, so on this configuration it does not fit at all; you would be raising -ncmoe to pay for it, before any throughput has arrived.

And the return is unknown. Unsloth's published guidance for this class of model is that "dense models are much more accelerated with MTP (1.4-2x) vs MoE models (1.15-1.25x)" (Qwen3.6 docs) — 1.15–1.25× at best, before offload is counted. Under offload even that is contested: an owner describing their box as "rtx 3060 12G+96G RAM, llama.cpp b9213" running a different model of this architecture at --n-cpu-moe 25 with the MTP flags on reports "Absolutely same decoding speed as on ordinary Q4_K_M GGUF" (discussion #18 on that model's GGUF repo), and a second community member there offers the reason: "CPU MoE seems to neuter any performance improvements that MTP adds." Neither is an org member and neither measured this model.

⚠️ And there is a crash report in this area worth knowing about. llama.cpp #26558 records llama-server aborting with a cublasSgemm CUDA_ERROR_INVALID_VALUE under --spec-type draft-mtp with parallel load and a saturated KV cache, on an RTX 4090 with CMAKE_CUDA_ARCHITECTURES=89 — the same compute capability as this card. Read its close carefully: it was closed as completed on 2026-08-28 with the comment "Closing as it no longer repros apparently", after the issue's own author reported the crash gone following a CUDA toolkit upgrade — that is a lapsed reproduction, not an identified fix. The thread does name a candidate: a collaborator points at PR #26574 ('ggml-cuda: provide static workspace for cuBLAS handles', merged 2026-08-20), which "should fix a cudaGraph-associated memory-leak" for CUDA toolkits below 12.4 — offered as a possible cause, not established as this crash's. The reported condition is parallel load, which -np 1 is not, so it is a hazard to recognise rather than one this recipe is exposed to.

You would rather load a smaller quant than offload at all

That is a real alternative and one publisher recommends it: abenzerps's hardware table names "IQ2_M or IQ3_M" for a 16 GB card, with "IQ2_M full offload; IQ3_M may require hybrid offload". Weigh it against the rest of that card, which is candid: "IQ1_M is a 1.75-bit-per-weight format intended for severe memory constraints; use IQ2_M or higher when possible." and "Prefer Q4_K_M or higher when system memory permits."

Check the arithmetic before taking it. Against this page's 13.917 GiB ceiling, IQ2_M at 11.682 GiB in the lead repo leaves room for a 131,072-token q8_0 cache and the recurrent state with 0.846 GiB to spare, so it genuinely escapes offload — and it is a 2-bit quantisation of a model whose routed experts are 512 wide, used for tool calling where a malformed argument is a failed task. Q2_K_L at 12.650 GiB does the same at 65,536 tokens. Below Q4 the trade on this card is quality for no -ncmoe, which is a cleaner trade than the 12 GB tier gets, and it is still a large quality bet to avoid a memory purchase.

Choosing a different quant publisher

Conversions of this model are still appearing: an enumeration on 2026-08-30 returned 22 repositories matching Apodex-1.1-mini across 11 publishers, one of which appeared that same day. Re-enumerate before you decide, and check the byte count and the file list rather than the tier name — across publishers a file named Q4_K_M spans more than a gigabyte for the same nominal tier. Within bartowski's own ladder the rungs adjacent to the lead are Q4_K_S at 19.619 GiB and IQ4_XS at 17.955 GiB; each buys back host RAM rather than VRAM, which on this card is the resource you have most of.

You wanted the vendor's own quantised weights

Apodex AI publishes -NVFP4, -GPTQ-Int4 and -FP8 repos, but they are vLLM/SGLang safetensors rather than GGUF and llama.cpp cannot load them at all. They are also far too large for this card before anything else is counted — 22.548 GiB and 22.958 GiB of weights for the first two, against 16 GiB — and the -NVFP4 repo is not a 4-bit checkpoint anyway: its hf_quant_config.json declares quant_algo: MIXED_PRECISION over 290 quantized layers, of which 250 are FP8 and only 40 — the routed experts — are NVFP4. Note that their Quick Starts launch on a single GPU: --tp-size 1 on both -NVFP4 and -GPTQ-Int4 and --tp 1 on -FP8; the --tp 8 and --tensor-parallel-size 8 lines belong to the unquantised base repo alone. For one RTX 4060 Ti the GGUF route above is the path.

Other pages on this exact card worth reading

Qwen3.8-27B on RTX 4060 Ti 16GB is a dense model of similar size that fits entirely on the card, and it is the useful contrast. Its own budget, as that page stood on 2026-08-30, puts its per-token VRAM read at about 13.4 GB against this recipe's 2.0–3.4, because a dense model has no experts to skip. That is why an offloaded 36B can plausibly out-decode a resident 27B on the same board. MiniMax H3 on RTX 4060 Ti 16GB is the page that works the host link on this card in detail. And if you have 24 GB and landed here by accident, Apodex 1.1 mini on RTX 3090, published 2026-08-28, fits the whole model on the card at the same 131,072-token context with none of this page's arithmetic.

The model emits garbage, or stops immediately on a very long prompt

Two different things, and only the second is architecture-specific.

For uniform-probability gibberish, check which backend the report you found is about. The nearest to this recipe is issue #25857, filed 2026-07-18 on a Windows 12 GB NVIDIA box: every token at an identical log-probability at any offload setting. Read its scope first — it is a different model (a 64-block Qwen3.6 variant, where this one has 41), at builds b9994 and b10066, both older than the b10630 this page pins, and it remains open with the stale label and zero comments as of 2026-08-30. Rebuild at a current release before inheriting the symptom.

For a completion that comes back empty with a clean HTTP 200 after a very long prompt, see issue #27756, which documents exactly that failure mode on a 64-layer sibling architecture beyond roughly 98–130k positions, reproduced on CUDA, on CPU and on a fork. As noted above, the same reporter's control run passed at 243k tokens on this model's 30-layer base architecture — so this window is not known to be affected, and that is the strongest statement the evidence supports rather than a guarantee. If you hit it, the signature is tokens_predicted: 1 with stop_type: "eos" and empty content, and it is worth telling us as well as the tracker.

For completeness on the issue this catalogue's older Apodex pages point at: llama.cpp #25717, the CUDA illegal-memory-access on vision requests with Qwen3.5-35B-A3B plus an mmproj, was closed as not_planned on 2026-08-30T01:13:13Z with the stale label. It was open when this catalogue's earlier Apodex pages were written in August 2026; its state was re-read on 2026-08-30 and it is closed. It never applied to a text-only run, which is what --no-mmproj makes this one.

No Ollama tag

There is no Ollama library entry for this model: ollama.com/library/apodex and /apodex-1.1-mini both return 404, and the registry.ollama.ai manifest for apodex returns 404 as well — checked 2026-08-30, with ollama.com/library/qwen3.5 and the registry.ollama.ai manifest for qwen3 both returning 200 as controls. Use llama.cpp directly. If a tag appears later, please tell us.