Nanbeige4.2-3B on RTX 3090 Ti: the full 256K context in VRAM with llama.cpp

What You'll Build

A llama-server endpoint running Nanbeige4.2-3B at 262,144 tokens of context, entirely in the RTX 3090 Ti's VRAM, on near-lossless Q8_0 weights with a q4_0 KV cache. Twenty-four gigabytes is the capacity where this model's declared context stops being a spec-sheet number, and the reason it takes 24 GB for a 3B model is an architectural quirk that doubles the cache.

Hardware data: RTX 3090 Ti (24 GB VRAM) · Q8_0 weights 4.130 GiB + q4_0 KV cache at 262,144 tokens 12.375 GiB + reserved logits 0.317 GiB + FlashAttention dequant scratch 1.000 GiB = 17.822 GiB accounted · no benchmark submitted yet · See benchmark data

⚠️ The 3B's weights are not the constraint here — the KV cache is. Nanbeige4.2-3B is a Looped Transformer: the model card says its "Looped Transformer architecture reuses the transformer layers to increase model capacity without adding parameters." config.json sets num_loops: 2, so its 22 blocks execute twice per forward pass over one shared set of weights — and each pass keeps its own keys and values. llama.cpp therefore allocates KV for 44 logical layers, and every context figure you would compute from num_hidden_layers: 22 is exactly half the truth. At f16 the full 262,144-token cache is 44.000 GiB, nearly twice this card. Everything below exists to get around that.

Requirements

Component	Minimum	This recipe
GPU	24 GB VRAM, CUDA	— not measured; the budget below is derived from file bytes and llama.cpp's allocation rules (/contribute)
RAM	8 GB system RAM	—
Storage	4.43 GB for the Q8_0 GGUF (decimal, as HuggingFace lists it)	4,434,787,488 bytes from the HF tree API; ~13 GB more if you also clone and build llama.cpp
Software	llama.cpp b10153+, CUDA toolkit, CMake	—

The build target: confirm it, do not inherit it

The RTX 3090 Ti and the RTX 3090 are different boards, and a sibling recipe's build flags are not evidence about this one. NVIDIA's own CUDA GPUs table settles it: GeForce RTX 3090 Ti and GeForce RTX 3090 are listed in the same compute-capability row, 8.6. So sm_86 is this card's native target, the toolkit floor is whatever your distribution ships for Ampere, and no part of the CUDA build below changes between the two boards.

The same page also settles the memory question, and it is worth stating because "24 GB" is doing a lot of work on this site. NVIDIA's RTX 3090 / 3090 Ti spec table gives both cards a Standard Memory Config of 24 GB GDDR6X on a 384-bit Memory Interface Width, and a different boost clock — 1.86 GHz on the Ti against 1.70 GHz. The budget below is byte-identical on the two boards because it is made of file sizes and allocation rules, not clocks. NVIDIA does not publish a memory-bandwidth figure on that page, and decode from a quantized cache is bandwidth-bound rather than clock-bound, so do not read the boost-clock difference as a throughput difference. Nobody has measured either card on this model; see Results.

Where the parameter counts come from

The card says "3B"; the repository is 8.34 GB in bf16, which is not what a 3B looks like. Both are correct, and the difference decides the budget, so here is the census rather than an assertion.

HuggingFace's tensor index for Nanbeige/Nanbeige4.2-3B reports 4,169,800,704 parameters, all BF16. That reconciles exactly with config.json:

• Untied embeddings (tie_word_embeddings: false): 166144 × 3072 × 2 = 1,020,788,736
• Per block: attention 3072×6144 + 2×(3072×1024) + 6144×3072 = 44,040,192, plus MLP 3 × 3072 × 10752 = 99,090,432, plus two RMS norms = 143,136,768
• 22 blocks plus the final norm = 3,149,011,968
• Total: 1,020,788,736 + 3,149,011,968 = 4,169,800,704 — the HF figure, to the parameter

So "3B" is the non-embedding count, which is how the model card's own comparison table labels it. The consequence that matters: the 22 blocks appear once in that census. If the two loop passes had their own weights the model would be roughly 7.3B. They do not — the loop reuses one stack, so you pay for the weights once and for the cache twice.

The KV cache doubles, and that is not a bug

1. The vendor designed it that way. A Nanbeige team member, replying on a thread titled "the modal is tiny but kv cache exploding!" — "We have also investigated KV-cache sharing across loop passes, but the performance gains were notably smaller than with the full looped setup." (discussion #10, leran1995, flagged as an org member), and again on discussion #18: "we did try sharing the KV cache across loop passes, but it noticeably hurt performance, so we kept the full cache in Nanbeige4.2". Sharing was built, measured and rejected.

2. llama.cpp implements it that way. src/models/nanbeige.cpp expands the logical layer count before anything is allocated — hparams.n_layer_all = (uint32_t) ((size_t) n_layer_phys * (size_t) n_loops);, under a comment reading "Expand logical layer count before load_tensors() allocates layers / KV." — and aliases the layer structs across loops under a second comment, "Share physical weights across loops; each slot still has its own KV index." src/llama-kv-cache.cpp then sizes the cache with const uint32_t n_layer = hparams.n_layer_all;. For nanbeige with num_loops: 2 that is 44, not 22.

3. Your file has to carry the flag. The loader reads the key as optional with a default of 1 — uint32_t n_loops_u = 1; ml.get_key(LLM_KV_NUM_LOOPS, n_loops_u, false); — so a GGUF converted by a path that omits it loads happily and runs 22 layers. That is a silently different model, not an error. llama_kv_cache prints the layer count in its startup line; confirm 44 layers before trusting anything below.

llama_kv_cache: size = ... MiB (262144 cells,  44 layers,  4/1 seqs), K (q4_0): ..., V (q4_0): ...

If it says 22 layers, your cache is half-size and your context is wrong.

KV cost per token, and what 24 GB buys

config.json gives num_key_value_heads: 8 and head_dim: 128, so each logical layer stores 8 × 128 = 1024 elements for K and 1024 for V per token. Over 44 logical layers that is 90,112 elements per token. Per-element cost comes from the block layouts in ggml/src/ggml-common.h, where QK4_0 and QK8_0 are both 32 values per block:

Cache type	Bytes per element	Bytes per token
f16 (default)	2	180,224 (176.0 KiB)
q8_0	34/32 = 1.0625	95,744 (93.5 KiB)
q4_0	18/32 = 0.5625	50,688 (49.5 KiB)

Multiplying out:

Context	f16 KV	q8_0 KV	q4_0 KV
32,768	5.500 GiB	2.922 GiB	1.547 GiB
65,536	11.000 GiB	5.844 GiB	3.094 GiB
131,072	22.000 GiB	11.688 GiB	6.188 GiB
262,144	44.000 GiB	23.375 GiB	12.375 GiB

Only the q4_0 column reaches the declared context on a 24 GiB card.

Two more terms, before you add weights

A flat 0.317 GiB of reserved logits. llama.cpp sizes one worst-case graph at startup. src/llama-context.cpp sets the logit rows that graph must hold to n_outputs_pp = std::min(n_tokens, cparams.n_outputs_max), where n_tokens is itself std::min(cparams.n_ctx, cparams.n_ubatch) and n_outputs_max defaults to n_batch. At stock settings that is 512 rows regardless of -c, each a full f32 distribution over this model's 166,144-token vocabulary:

512 rows × 166,144 vocab × 4 B = 340,262,912 B = 324.5 MiB = 0.317 GiB

It is large only because the vocabulary is, and the dial that moves it is -ub, not -c — so it is the same 0.317 GiB in every row of the budget table.

4,096 bytes per context token of FlashAttention dequant scratch — but only on a quantized cache. The kernels read f16, so a quantized cache is expanded into scratch VRAM first. ggml/src/ggml-cuda/fattn.cu sets need_f16_K = true; need_f16_V = true; for the TILE and MMA_F16 kernels, and fattn-common.cuh then appends an f16 copy of each to the attention output tensor's own allocation:

    if (need_f16_K && K->type != GGML_TYPE_F16) {
        data.end = GGML_PAD(data.end, 128);
        data.K   = data.end;
        data.end += ggml_nelements(K)*ggml_type_size(GGML_TYPE_F16);
    }

with a matching block for V. Two properties decide the size. The guard's second conjunct means an f16 cache pays exactly zero — this is the one place where the default cache is the cheaper one. And the term is charged at the reserve, not only during prompt processing: llama.cpp sizes its worst-case graph at n_tokens = min(n_ctx, n_ubatch) = 512, and on this card the vector kernel — the one that reads a quantized cache directly and needs no scratch — is selected only when Q->ne[1] == 1. A 512-row batch fails that test, so the dequantizing path is what gets budgeted.

For this model the cost is (1024 + 1024) × 2 B = 4,096 bytes per token of -c: 0.250 GiB at 65,536, 0.500 GiB at 131,072 and 1.000 GiB at 262,144.

Add all four terms. Both weight tiers are from the bartowski GGUF repo, byte counts via the HuggingFace tree API:

Configuration	Weights	KV	Logits	FA scratch	Accounted total	Slack on 24 GiB
Q8_0 · q4_0 KV · 262,144 ctx (this recipe)	4.130	12.375	0.317	1.000	17.822 GiB	6.178 GiB
Q8_0 · q8_0 KV · 131,072 ctx	4.130	11.688	0.317	0.500	16.635 GiB	7.365 GiB
Q8_0 · f16 KV · 65,536 ctx	4.130	11.000	0.317	0.000	15.447 GiB	8.553 GiB
Q4_K_M · f16 KV · 65,536 ctx	2.500	11.000	0.317	0.000	13.817 GiB	10.183 GiB

The 6.178 GiB of slack is not spare. It absorbs the CUDA context, the rest of the compute buffer for a 44-layer unrolled forward pass, and the framebuffer if this card also drives your displays — none of which is measured on this GPU, which is why the recipe stops at the declared context rather than arguing for more.

Separately, budget 256 MiB of system RAM for the attention mask, which is deliberately absent from every figure above because it is not VRAM: src/llama-graph.cpp carries GGML_ASSERT(ggml_backend_buffer_is_host(self_kq_mask->buffer));, so it is a pinned host allocation. At this context it is n_kv × n_ubatch × 2 B = 262,144 × 512 × 2 = 268,435,456 B. Counting it as VRAM is a common way to over-budget this model.

Why Q8_0 weights and not Q4_K_M

On an 8 GB card the weight tier is the whole decision. On 24 GB it barely registers, and the arithmetic says so. Within the bartowski repo, Q8_0 (4,434,787,488 B) costs 1.631 GiB more than Q4_K_M (2,684,023,968 B). Spent on cache instead, that buys 31,957 more tokens at q4_0 or 17,535 at q8_0 — 12% and 7% of the 262,144 ceiling.

Both figures use the effective per-token price of context, not the sticker KV price: raising -c by one token on a quantized cache costs the cache bytes plus 4,096 bytes of dequant scratch, so the real rate is 54,784 B/token at q4_0 and 99,840 B/token at q8_0. On the sticker rates alone the same 1.631 GiB would read 34,540 and 18,285, and the gap between the two pairs is the dequant tax. The question the comparison asks is what the freed memory buys if spent on context, and raising the context drags the scratch along with it — so the sticker rate answers a question nobody is asking.

Trading a near-lossless weight tier for at most 13% more context, on a model whose entire pitch is agentic and reasoning accuracy at 3B, is the wrong way round. Quantizers differ and their bytes are not interchangeable: owao's Q8_0 is 4,434,787,168 B — 320 bytes apart, same to three decimals — but its Q4_K_M is a different file at 2,574,807,904 B. owao ships a 13-rung ladder and bartowski a 23-rung one; pick one repo and do not mix their numbers in one sum.

Installation

1. Build llama.cpp with CUDA

Mainline support arrived in PR #25994, merged 2026-07-27 as commit b77d646751d01c0962bc203b6809e9d94f7d50b7, and release b10153 carries exactly that commit — so b10153 is the first tagged build that will load this architecture. Anything older refuses it.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

That relies on llama.cpp's native-architecture detection. If nvcc cannot see the card — a container build, a cross-compile — docs/build.md documents naming the capability explicitly, and for this board that is 86:

cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="86" -DCMAKE_BUILD_TYPE=Release

The model card still tells you to build the vendor's own fork (git clone -b nanbeige42 https://github.com/Nanbeige/llama.cpp.git). That was correct at release and is not any more — mainline carries the architecture, and mainline is where the fixes land.

2. Download the weights

Nanbeige publishes no first-party GGUF. Enumerating the org returns ten repositories and not one of them carries the gguf tag; the quantized builds it does publish are FP8 and GPTQ-Int8. A team member said on discussion #1 that official quantized versions were being prepared, which as of this writing has not happened. Everything below uses a community conversion.

pip install -U huggingface_hub

hf download bartowski/Nanbeige_Nanbeige4.2-3B-GGUF \
  --include "Nanbeige_Nanbeige4.2-3B-Q8_0.gguf" \
  --local-dir ./nanbeige4.2-3b

Check provenance on whichever repo you pick, because this model has a date cliff. The canonical repo landed a tokenizer commit on 2026-07-27, hours before mainline support merged the same day, and a quant is a frozen copy of upstream as of the moment it was uploaded. bartowski's card states the quants were built with llama.cpp release b10159, and the HuggingFace API gives that repo createdAt 2026-07-28T14:38Z — comfortably after both events. owao's repo has an earlier createdAt (2026-07-21T20:33Z) but a lastModified of 2026-07-29T13:06Z, so check the individual file's commit before assuming a given rung there is post-merge.

Running

The long-context configuration

./build/bin/llama-server \
  -m ./nanbeige4.2-3b/Nanbeige_Nanbeige4.2-3B-Q8_0.gguf \
  --host 127.0.0.1 --port 8080 \
  -ngl 99 \
  -c 262144 \
  -fa on \
  -ctk q4_0 -ctv q4_0 \
  --temp 0.6 --top-p 0.95 --top-k 20

That is the full declared context — "The model supports a context length of up to 262,144 tokens (256K)." — resident in VRAM on one card, at 17.822 GiB accounted.

Four of those flags are load-bearing:

• -c 262144 — set it explicitly, and never -c 0. Omitting -c is not the same as passing 0: with -c unset, llama.cpp's -fit on default (common/fit.cpp) interpolates the context down until the model fits free memory with a margin, so you silently get some arbitrary reduced number. With -c 0, common/arg.cpp sets params.fit_params_min_ctx = UINT32_MAX; — a deliberate "give me the full trained context, do not shrink it" — and you get 262,144 cells with the f16 default cache, i.e. 44.000 GiB, i.e. an immediate out-of-memory abort.
• -fa on. A quantized V cache requires FlashAttention; src/llama-context.cpp throws quantized V cache was requested, but this requires Flash Attention otherwise. The auto default will enable it for you, but being explicit makes the failure mode legible.
• -ctk q4_0 -ctv q4_0 — matched types, not mixed. A stock CUDA build instantiates FlashAttention kernels only for identical K and V types. fattn.cu guards it literally — #ifndef GGML_CUDA_FA_ALL_QUANTS / if (K->type != V->type) { / return BEST_FATTN_KERNEL_NONE; — and the default instance list compiles exactly four vector pairs: f16-f16, q4_0-q4_0, q8_0-q8_0, bf16-bf16. A clever-looking -ctk q8_0 -ctv q4_0 needs a rebuild with -DGGML_CUDA_FA_ALL_QUANTS=ON.
• Leave --parallel alone. llama-server defaults it to -1, and tools/server/server.cpp resolves the sentinel with params.n_parallel = 4; and params.kv_unified = true; in the same branch. Unified means one shared cache of -c cells that any single slot may consume in full — so the table above is the whole KV bill, and one conversation can still reach 262,144. Passing -np 4 explicitly skips that branch, leaves kv_unified false, and src/llama-context.cpp then computes cparams.n_ctx_seq = cparams.n_ctx / cparams.n_seq_max; — same total memory, a quarter of the context per conversation. If you want one slot, say -np 1, not -np 4.

One Ampere-specific detail worth knowing

fattn.cu branches on compute capability inside the quantized-cache path. On a card below GGML_CUDA_CC_ADA_LOVELACE — and 8.6 is below the 8.9 that constant names — the direct-read vector kernel is selected only when Q->ne[1] == 1, i.e. a single token in flight; an Ada card takes it up to two. With llama-server's default of four slots on a unified cache, a moment where two slots each contribute one token therefore dispatches differently here than it would on a newer 24 GB card.

This changes nothing in the budget above — the reserve graph is sized at 512 rows on every card, so the dequant scratch is charged either way — and it is a statement about kernel selection, not about speed. No throughput measurement exists on this card to quantify it.

The quality-first alternative

If you would rather not run a 4-bit K cache, halve the context and keep the near-lossless one. 16.635 GiB accounted, and 131,072 is exactly the max-new-tokens the model card recommends for reasoning and chat:

./build/bin/llama-server \
  -m ./nanbeige4.2-3b/Nanbeige_Nanbeige4.2-3B-Q8_0.gguf \
  --host 127.0.0.1 --port 8080 \
  -ngl 99 \
  -c 131072 \
  -fa on \
  -ctk q8_0 -ctv q8_0 \
  --temp 0.6 --top-p 0.95 --top-k 20

For agentic and tool-use work the card's sampling table recommends --temp 1.0 with 65,536 new tokens instead.

Spend the headroom on preserve_thinking

The chat template takes preserve_thinking, which controls whether reasoning from earlier assistant turns stays in the context. It is a KV-versus-quality dial, and on this card you are on the right side of it — a Nanbeige team member on discussion #9: "In our evaluations, enabling preserve_thinking generally provides better performance and better kv-cache reuse, so we recommend keeping it enabled when context length and memory allow." Pass it through llama-server's OpenAI-compatible endpoint as "chat_template_kwargs": {"preserve_thinking": true}.

Results

• Speed: omitted — there is no measurement of this model on an RTX 3090 Ti. /check/nanbeige4-2-3b/rtx-3090-ti has zero benchmarks; all 28 discussion threads on the canonical repo were fetched individually and none mentions a 3090 Ti, a 3090 or any 24 GB card; and llama.cpp's tracker returns no Nanbeige report on this GPU. The only throughput datapoint that exists anywhere for this model is a community report on an 8 GB RTX 5060, posted by thermi6 on discussion #18 — a different capacity on a newer generation, which constrains this card in neither direction, so its figures are not repeated here and nothing is interpolated from them. If you run this, please contribute your measurement so the next reader gets a real figure.
• VRAM usage: 17.822 GiB accounted at the lead configuration — 4.130 GiB weights, 12.375 GiB cache, 0.317 GiB reserved logits and 1.000 GiB of FlashAttention dequant scratch — leaving 6.178 GiB of the card's 24 GiB. The q8_0/131,072 alternative comes to 16.635 GiB. Both derived, not measured: see /check/nanbeige4-2-3b/rtx-3090-ti. Separately, budget ~256 MiB of system RAM for the pinned attention mask.
• Quality notes: I found no evaluation of this model under a quantized KV cache at any tier in the spaces searched — the model card, all 28 canonical discussion threads, both GGUF repos' cards, and llama.cpp's issue and PR search. Treat that as a gap, not as evidence the loss is small. q8_0 cache is generally treated as near-lossless across the llama.cpp ecosystem; q4_0 is the aggressive rung, and the reason the alternative configuration above exists. The vendor publishes no throughput figures at all — the card carries quality benchmarks and a sampling table and nothing about speed.
• Context is trained, not extrapolated: rope_scaling is null in config.json, so 262,144 is a training target rather than a RoPE-scaling claim.

For the full benchmark data, see /check/nanbeige4-2-3b/rtx-3090-ti.

Troubleshooting

llama_model_load: error loading model architecture: unknown model architecture: 'nanbeige'

Your build predates b10153. Rebuild from mainline master, or use a release tag at or above b10153 — that build's commit is the merge of PR #25994. On discussion #23 one user reports still being unable to load the architecture, and a different user then reports that mainline release b10199 loads it. Distribution channels lag: the model card itself notes that LM Studio's bundled llama-server does not support nanbeige, and tells you to copy your own build's binaries into the LM Studio backend directory.

Out of memory at startup with the default cache type

You almost certainly left -ctk/-ctv at f16. At 262,144 tokens that is 44.000 GiB of cache — the arithmetic is in the table above, and it is the single most common way to be surprised by this model. Quantize the cache, or drop to -c 65536, where f16 fits at 15.447 GiB accounted and pays no dequant scratch at all, which is why it lands closer to the quantized options than the cache column alone suggests.

If you need the full context on a card that genuinely cannot hold the cache, --no-kv-offload moves the cache to system RAM and keeps the weights on the GPU. On a 24 GB card you should not need the flag.

Tool calls come back as plain text instead of executing

Fixed upstream, in release tag b10227 — check your build before you debug anything else. The defect was in llama.cpp rather than in your configuration: the model sometimes emits <tool_call> followed by a space rather than a newline, and the chat parser matched the marker with the newline attached, so the call was returned verbatim as content. The author of PR #26324 put the pre-fix rate at roughly 25%, and the same defect was reported independently on the model's own discussion #17, whose author measured 24 of 29 calls parsing before his patch and 29 of 29 after.

The route to the fix is worth a sentence, because #26324 is still the first thing a search turns up and it never merged. A llama.cpp maintainer pushed back on the proposed whitespace trimming — "trimming whitespaces for some of the Qwen models was really detrimental to their output" — and another contributor pointed instead at the grammar trigger, and at PR #26252. That one, a specialized parser, merged on 2026-08-02 and went out as b10227; on 2026-08-10 #26324's author closed his own pull request unmerged, reporting that tool calls now worked every time. Tool calling therefore has a build floor of its own — b10227, above the b10153 this page requires to load the model at all. If calls still come back as text your build sits between the two; rebuild.

HTTP 400 — Failed to initialize samplers on a json_schema request

Any request with a response_format of json_schema fails at sampler init on the default jinja chat path, before a token is generated. Two users reproduced it independently on discussion #6, on different builds and different quant tiers, and both found the same workaround: add --no-jinja to the llama-server command.

./build/bin/llama-server -m ./nanbeige4.2-3b/Nanbeige_Nanbeige4.2-3B-Q8_0.gguf --no-jinja ...

Overriding the surface template with --chat-template chatml does not help, and the raw /completion endpoint with the same schema returns conformant JSON — which isolates the fault to the chat-completions grammar trigger rather than the model or the grammar. The cost of --no-jinja is that you lose the model's own chat template, so apply the prompt format yourself if you take this route.

ollama run nanbeige/nanbeige4.2:3b-Q4_K_M returns not found

The model card lists that command under its Ollama section, but the registry entry it names does not exist, and neither do the library/ paths for either spelling of the name. The card's other Ollama route — building Ollama from the vendor's fork and copying a llama.cpp build into its runtime payload directory — is a lot of machinery to reach a llama-server you already have. Use llama-server directly.

Mismatched -ctk / -ctv silently disables FlashAttention

See the flag notes under Running. A stock CUDA build compiles only the four matched vector instances; an asymmetric cache needs -DGGML_CUDA_FA_ALL_QUANTS=ON. The symptom is not an error message — the dispatcher returns BEST_FATTN_KERNEL_NONE and you lose the kernel you were counting on.

Anything else, or a real throughput measurement on this card, is welcome via the submission form.