Nanbeige4.2-3B on RTX 3060 Ti: 32K Context in 8 GB, and What sm_86 Does Not Change

What You'll Build

A local llama-server endpoint running Nanbeige4.2-3B on one RTX 3060 Ti, with a 32,768-token window and an 8-bit KV cache, entirely resident in the card's 8 GB. The weights are the easy part on a 3B model. The whole engineering question on this card is how much context you can buy with what is left, and this model charges for context at exactly twice the rate its config file implies.

Hardware data: RTX 3060 Ti (8 GB VRAM) · Q4_K_M weights 2.398 GiB + q8_0 KV at 32,768 tokens 2.922 GiB + reserved logits 0.317 GiB + FlashAttention dequant scratch 0.125 GiB = 5.762 GiB accounted · no benchmark submitted yet · See benchmark data

⚠️ The KV cache is sized for 44 layers, not the 22 in config.json. Nanbeige4.2-3B is a Looped Transformer: the model card says "Its Looped Transformer architecture reuses the transformer layers to increase model capacity without adding parameters.", and config.json sets num_loops: 2 alongside num_hidden_layers: 22. The 22 blocks execute twice per forward pass over one shared set of weights, and each pass keeps its own keys and values. llama.cpp does the expansion before it allocates anything — src/models/nanbeige.cpp sets hparams.n_layer_all = n_layer_phys * n_loops under the comment "Expand logical layer count before load_tensors() allocates layers / KV.", and aliases the weights a few lines later with "Share physical weights across loops; each slot still has its own KV index." — and src/llama-kv-cache.cpp then sizes the cache with const uint32_t n_layer = hparams.n_layer_all;. Every context figure computed from 22 layers is exactly half the truth.

The team confirms the cost is deliberate. On discussion #18 a Nanbeige member writes: "we did try sharing the KV cache across loop passes, but it noticeably hurt performance, so we kept the full cache in Nanbeige4.2".

Requirements

Component	Minimum	This recipe
GPU	8 GB VRAM, CUDA compute capability 7.5+	RTX 3060 Ti (8 GB) — not measured; the budget below is derived from file bytes and llama.cpp's own allocation rules (/contribute)
RAM	8 GB system RAM	— (16 GB+ for the --no-kv-offload long-context mode)
Storage	2.57 GB for the Q4_K_M GGUF (decimal, as HuggingFace lists it)	2,574,807,904 bytes, from the HF tree API
Software	llama.cpp b10153+, CUDA toolkit 11.1+, CMake 3.18+	—

This is the lowest toolkit floor of the three CUDA generations llama.cpp annotates — not the lowest floor that exists, since the 75-virtual and 80-virtual targets the same file always appends carry no version gate at all. It is stated in llama.cpp's own build logic: ggml/src/ggml-cuda/CMakeLists.txt annotates each architecture it can target:

# 86     == RTX 3000, needs CUDA v11.1
# 89     == RTX 4000, needs CUDA v11.8
# 120    == Blackwell, needs CUDA v12.8, FP4 tensor cores

NVIDIA's CUDA GPU Compute Capability table lists the GeForce RTX 3060 Ti at 8.6, so 86 is your row: any CUDA 11.1-or-newer toolkit can compile for this card, and you never name an architecture yourself. Which mechanism covers you depends on where you sit: the native path is gated on CUDAToolkit_VERSION VERSION_GREATER_EQUAL "11.6" AND CMAKE_VERSION VERSION_GREATER_EQUAL "3.24", so at the 11.1 toolkit floor above — or on the CMake 3.18 this recipe allows — it does not run and the fallback list runs instead. That list appends 86-real unconditionally, so the outcome is the same either way.

Do not follow the model card's llama.cpp instructions. The card still tells you to git clone -b nanbeige42 https://github.com/Nanbeige/llama.cpp.git. That was correct at release. Mainline merged native nanbeige support in PR #25994 on 2026-07-27, and issue #26086 tracking the request was closed the same day. Release tag b10153 is that merge commit, so it is the earliest tag that works — anything newer is fine.

Installation

1. Build llama.cpp with CUDA

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j $(nproc)

Confirm the checkout actually carries the architecture — this file does not exist in builds older than 2026-07-27:

test -f src/models/nanbeige.cpp && echo "nanbeige support present"

2. Download the Q4_K_M GGUF

Nanbeige publishes no first-party GGUF: the org's ten public repositories include -FP8 and -GPTQ-Int8 builds of this model and nothing in GGUF form. This recipe uses owao/Nanbeige4.2-3B-GGUF; bartowski/Nanbeige_Nanbeige4.2-3B-GGUF is a wider alternative ladder. Pick one repository and stay inside it — the two quantizers' files are not byte-identical (their Q4_K_M builds differ by 109 MB), and mixing their numbers into one budget is how a sum goes quietly wrong.

pip install -U huggingface_hub numpy

hf download owao/Nanbeige4.2-3B-GGUF \
  Nanbeige4.2-3B-Q4_K_M.gguf \
  --local-dir ./models

numpy is not a dependency of huggingface_hub, and the next step needs it.

3. Verify the artifact carries the loop parameter

Worth doing by hand exactly once. If a GGUF was produced without num_loops, llama.cpp defaults the key to 1, runs 22 layers instead of 44, and you get a silently different model — no error, no warning, half the depth.

python gguf-py/gguf/scripts/gguf_dump.py --no-tensors \
  ./models/Nanbeige4.2-3B-Q4_K_M.gguf | grep -E "num_loops|block_count"

You want two lines, ending = 22 and = 2 respectively:

     18: UINT32     |        1 | nanbeige.block_count = 22
     30: UINT32     |        1 | nanbeige.num_loops = 2

A missing num_loops line is the failure case. (The leading index numbers vary from file to file; only the key names and the values matter.)

Two commands that look like they should do this job do not, so do not substitute them. llama-gguf <file> r n prints key names only, never their values. And llama-cli suppresses the loader's metadata dump at its default verbosity, besides being an interactive chat client that will sit waiting for input rather than exiting.

Running

./build/bin/llama-server \
  -m ./models/Nanbeige4.2-3B-Q4_K_M.gguf \
  --host 127.0.0.1 --port 8080 \
  -ngl 99 \
  -c 32768 \
  -fa on \
  -ctk q8_0 -ctv q8_0 \
  --temp 1.0 --top-p 0.95 --top-k 20

Open http://127.0.0.1:8080 for the built-in chat UI, or point any OpenAI-compatible client at http://127.0.0.1:8080/v1. --temp 1.0 is the model card's recommendation for agentic and tool-use work; it recommends --temp 0.6 for reasoning and chat.

Three flags are load-bearing:

• -c 32768, set explicitly. Omitting it is not neutral — llama.cpp's context-fitting default interpolates the window downward until the model fits free memory, so you silently get some other number and the budget below stops describing your process.
• -ctk q8_0 -ctv q8_0 — matched types. A stock CUDA build instantiates FlashAttention kernels only for identical K and V types; ggml/src/ggml-cuda/fattn.cu returns BEST_FATTN_KERNEL_NONE when K->type != V->type outside GGML_CUDA_FA_ALL_QUANTS. A clever-looking -ctk q8_0 -ctv q4_0 needs a rebuild with -DGGML_CUDA_FA_ALL_QUANTS=ON.
• Leave --parallel alone. With no flag, tools/server/server.cpp resolves the default with params.n_parallel = 4; params.kv_unified = true; in the same branch (it tests params.n_parallel < 0). Unified means one shared pool of -c cells that a single conversation may consume in full, which is what the numbers below assume. Passing an explicit positive --parallel N skips that branch, leaves the cache non-unified, and gives each conversation -c / N for identical memory.

What sm_86 changes about this budget — and what it does not

It is tempting to explain the FlashAttention scratch term below as a property of Ampere. It is not, and getting that wrong would mislead anyone reading across cards, so here is the actual branch.

At startup llama.cpp reserves one worst-case prompt-processing graph. src/llama-context.cpp builds it at const uint32_t n_tokens = std::min(cparams.n_ctx, cparams.n_ubatch); — 512 rows at stock settings. In ggml_cuda_get_best_fattn_kernel, turing_mma_available(cc) is true for every NVIDIA card from compute capability 7.5 upward, so sm_86 enters the same branch as every newer architecture. Inside it, a quantized cache reaches the cheap vector kernel only at Q->ne[1] <= 2 (compute capability 8.9 and above) or at Q->ne[1] == 1 (below that), and the trailing fallback also wants a single row. A 512-row reserve fails all three tests, so the function returns BEST_FATTN_KERNEL_MMA_F16 — and ggml_cuda_flash_attn_ext_get_alloc_size sets need_f16_K = need_f16_V = true for MMA_F16 and TILE alike. There is no NVIDIA compute capability at which a 512-row reserve escapes those flags. The term comes from the shape of the reserved graph, not from your card.

What compute capability 8.6 does change is the two-row case at generation time: over a quantized cache, an 8.9-or-newer card keeps a two-row batch on the vector kernel while this one promotes it to the tensor-core kernel. That changes which kernel runs, not how much memory was reserved, so no figure on this page moves.

Where the numbers come from

llama.cpp sizes the cache as n_layer × (n_embd_k_gqa + n_embd_v_gqa) × bytes_per_element per token. config.json gives num_key_value_heads: 8 and head_dim: 128, so each layer stores 8 × 128 = 1024 elements for K and 1024 for V. Over 44 logical layers that is 90,112 elements per token:

Cache type	Bytes per element	Bytes per token
f16 (default)	2	180,224
q8_0	34/32 = 1.0625	95,744
q4_0	18/32 = 0.5625	50,688

The block sizes are sizeof(block_q8_0) = 34 bytes and sizeof(block_q4_0) = 18 bytes per 32 elements, from ggml/src/ggml-common.h. Multiplying out:

Context	f16 KV	q8_0 KV	q4_0 KV
16,384	2.750 GiB	1.461 GiB	0.773 GiB
32,768	5.500 GiB	2.922 GiB	1.547 GiB
65,536	11.000 GiB	5.844 GiB	3.094 GiB
262,144	44.000 GiB	23.375 GiB	12.375 GiB

Weights and cache are not the whole bill, and on 8 GB the other two terms decide rows.

A flat 0.317 GiB of reserved logits. src/llama-context.cpp sets the logit rows the reserved graph must hold to n_outputs_pp = std::min(n_tokens, cparams.n_outputs_max), where n_tokens is the 512 above and n_outputs_max defaults to n_batch. That is 512 rows, each a full f32 distribution over this model's unusually large 166,144-token vocabulary: 512 × 166,144 × 4 B = 340,262,912 B = 324.5 MiB. It does not move when you change -c; it halves when you halve -ub.

4,096 bytes per context token of FlashAttention dequant scratch — only because the cache is quantized. fattn-common.cuh adds ggml_nelements(K) * ggml_type_size(GGML_TYPE_F16) for K and again for V, each behind if (need_f16_K && K->type != GGML_TYPE_F16) — so an f16 cache pays exactly none of it, and a quantized one pays (1024 + 1024) × 2 B per token: 0.062 GiB at 16,384, 0.125 GiB at 32,768, 0.250 GiB at 65,536.

Adding all four terms, the practical envelope on 8 GiB:

Configuration	Weights	KV	Logits	FA scratch	Accounted total	Unallocated on 8 GiB
Q4_K_M · f16 · 32,768	2.398	5.500	0.317	0.000	8.215 GiB	will not fit
Q4_K_M · f16 · 16,384	2.398	2.750	0.317	0.000	5.465 GiB	2.535 GiB
Q4_K_M · q8_0 · 32,768 (this recipe)	2.398	2.922	0.317	0.125	5.762 GiB	2.238 GiB
Q5_K_M · q8_0 · 32,768	2.782	2.922	0.317	0.125	6.146 GiB	1.854 GiB
Q4_K_M · q4_0 · 65,536	2.398	3.094	0.317	0.250	6.059 GiB	1.941 GiB
Q4_K_M · q8_0 · 49,152	2.398	4.383	0.317	0.188	7.285 GiB	0.715 GiB — headless only
Q4_K_M · q8_0 · 65,536	2.398	5.844	0.317	0.250	8.809 GiB	will not fit

The first row is the trap: a reader who budgets from num_hidden_layers: 22 predicts 2.750 GiB of f16 cache at 32,768, concludes there is room to spare, and is wrong by a factor of two.

Two consumers sit on the same 8 GiB and are deliberately not in the accounted total, because nothing here measures them: the CUDA runtime's own per-process context, and your desktop compositor if this card drives a monitor. The 2.238 GiB left over is what they share. One more thing lives outside VRAM entirely — the attention mask is n_kv × n_ubatch in f16, and src/llama-graph.cpp asserts ggml_backend_buffer_is_host(self_kq_mask->buffer) on it, so it is pinned system RAM. Counting it as VRAM is a common way to over-budget this model.

Why this page leads 32,768 rather than the card's 65,536

The model card's own scenario table asks for 65,536 new tokens on agentic work, and the table above shows that window is reachable here — but only by dropping the cache to q4_0, and there is no published evaluation of this model under a quantized KV cache at any tier. On the oldest silicon in the 8 GB class the conservative half of that unmeasured trade is the better default: keep the 8-bit cache, take the smaller window, and step up to q4_0 at 65,536 deliberately if your workload actually needs the window. Both operating points are costed above; neither is a guess.

Results

• Speed: omitted — no measurement exists for this pair, and none exists for this model on any Ampere card. The only throughput figures anywhere for Nanbeige4.2-3B are community-reported on an RTX 5060: in discussion #18 user thermi6 posts a llama-server log for that card at 65,536 context with a q4_0 cache. That is a two-generations-newer architecture, and this page has no defensible way to scale it onto this one: the only difference between the two cards it has actually sourced is the FlashAttention kernel branch above (which kernel a two-row batch over a quantized cache takes), and that is far too narrow to carry a throughput estimate. His figures also fall steeply within that single session as the conversation grows, so they describe a session at least as much as they describe a card. Read them as evidence that this model runs in 8 GB at all, not as an estimate for this card; publishing a number derived from them would be inventing one. If you run this configuration, please contribute the measurement so /check/nanbeige4-2-3b/rtx-3060-ti stops being empty.
• VRAM usage: 5.762 GiB accounted at the lead configuration — 2.398 GiB of Q4_K_M weights, 2.922 GiB of q8_0 cache at 32,768 tokens, 0.317 GiB of reserved logits and 0.125 GiB of FlashAttention dequant scratch — leaving 2.238 GiB of the card's 8 GiB for the CUDA context, the activation working set and your display. Derived from measured file bytes and llama.cpp's own allocation rules, not measured on hardware.
• Quality notes: the model is ~4.17 B parameters total and ~3.15 B non-embedding — the "3B" in the name counts the non-embedding half, and the extra billion is the untied 166,144-token vocabulary at both ends of the stack. That vocabulary is also why the logits reservation above is as large as it is: the same 512-row slot would hold 64.0 MiB for a 32,768-token vocabulary and holds 324.5 MiB here.

Reaching the full 256K context

The card states that "The model supports a context length of up to 262,144 tokens (256K)." At 44 logical layers that cache is 12.375 GiB even at q4_0 — more than this entire card, before a single weight byte. It stays reachable by keeping the weights on the GPU and the cache in system RAM:

./build/bin/llama-server \
  -m ./models/Nanbeige4.2-3B-Q4_K_M.gguf \
  -ngl 99 -c 262144 -fa on \
  -ctk q4_0 -ctv q4_0 --no-kv-offload

This needs roughly 13 GB of free system RAM and trades decode speed for window size. Treat it as a capability, not a default.

For the full benchmark data, see /check/nanbeige4-2-3b/rtx-3060-ti.

Troubleshooting

llama-server OOMs at 32,768 tokens

Almost always the default f16 cache, which needs 5.500 GiB at this context before any weights — 8.215 GiB accounted against a card that has 8. Add -ctk q8_0 -ctv q8_0.

If the cache flags are already right and you are a couple of hundred MiB short, reach for -ub before you touch -c, because the logits reservation scales with the micro-batch and not with the window:

-ub	reserved logits	vs. default
512 (default)	0.317 GiB	—
256	0.158 GiB	−0.158 GiB
128	0.079 GiB	−0.238 GiB

It costs prompt-processing throughput, not context and not quality. Note it does not touch the 0.125 GiB of dequant scratch, which is sized by -c. Past that, drop to -c 16384 or move both cache types to q4_0.

unknown model architecture: 'nanbeige'

The build predates b10153. Rebuild from mainline master, or take a release tag at or above b10153. Distribution channels lag: the model card notes that LM Studio's bundled llama-server does not support nanbeige and tells you to copy your own build's binaries into the LM Studio backend directory.

Tool calls come back as plain text instead of executing

Fixed in mainline as of tag b10227 — check your build before debugging anything else. It was a llama.cpp parser bug, not a model bug and not your configuration: the model sometimes emits <tool_call> followed by a space rather than a newline, and the auto-generated parser required that newline verbatim, so the whole reply was consumed as content and the tool-call rule never ran. PR #26324 measured the loss at roughly a quarter of calls across 240 markers; the model's own discussion #17 reported roughly one call in six, though on the vendor nanbeige42 fork rather than on mainline. Both are community reports rather than a maintainer measurement, so read them as "some calls" rather than a constant. The fix arrived by a different route — PR #26252, a specialized parser merged on 2026-08-02 and released as tag b10227 — after which the author of #26324 reported tool calls working every time on a newer build and closed his own pull request unmerged on 2026-08-10. If calls still come back as text, your build predates b10227; rebuild. The failure bit hardest at the --temp 1.0 this recipe recommends for agentic work — #17's small probe found every generation parsing at temperature 0.

response_format: json_schema fails before any token is generated

Grammar-constrained decoding aborts at sampler init with Failed to initialize samplers: Unexpected empty grammar stack. Two users reproduced it independently on discussion #6 — but both were running the vendor fork rather than the mainline build installed here, so treat the status on mainline as untested rather than known-broken. If you hit it, restart the server with --no-jinja; plain generation is unaffected either way.

ollama pull cannot find the model

There is no entry in the official Ollama library, and the model card's own ./ollama run nanbeige/nanbeige4.2:3b-Q4_K_M names a namespace that returns 404 on ollama.com. Community re-uploads exist under user namespaces, but most search hits for "nanbeige" are 4.1 builds — a different model generation. Use llama-server as above.

The model loads but the output is subtly poor

Re-run step 3 and confirm the GGUF reports nanbeige.num_loops = 2. A file converted by a path that omits the key loads happily and runs at half depth. Also confirm the build post-dates 2026-07-27; older mainline builds refuse the architecture outright rather than misbehave.