Nanbeige4.2-3B on Apple M4 Max: 546 GB/s for a stack that streams twice

What You'll Build

A local OpenAI-compatible server running Nanbeige4.2-3B at its full 262,144-token context on an Apple M4 Max, via mainline llama.cpp with the Metal backend and a community GGUF. Two unusual properties of this model decide the whole page. Its 22 decoder blocks execute twice per forward pass, so it moves nearly twice the bytes per generated token that its file size implies — which is why the M4 Max's memory bandwidth, not its core count, is what you are buying here. And that same loop doubles the KV cache, which is why the full window needs a plan rather than just a flag.

Hardware data: Apple M4 Max (48 GB unified memory, 36.000 GiB GPU-addressable, 546 GB/s) · Q4_K_M weights 2.398 GiB + q8_0 KV at 262,144 tokens 23.375 GiB + reserved logits 0.317 GiB = 26.090 GiB derived · no Apple benchmark submitted yet · See benchmark data

⚠️ 44 cache layers, not 22. config.json sets num_loops: 2. The 22 physical blocks are executed twice and every pass keeps its own KV entries, so the cache is sized against 44 layers while the weights are stored once. llama.cpp says so in the architecture file itself: src/models/nanbeige.cpp widens the count — "Expand logical layer count before load_tensors() allocates layers / KV." — and the tensor loop is commented "Share physical weights across loops; each slot still has its own KV index." That is 176.0 KiB per token at f16, where the config file reads like 88.0 KiB.

Requirements

Component	Minimum	This recipe
GPU	Apple silicon, Metal	Apple M4 Max, 48 GB unified memory — not measured on this chip; every figure below is derived from file bytes and the runtime's own allocation rules (/contribute)
Unified memory	36 GB — the smallest Apple capacity whose default share clears the lead configuration	48 GB, of which 36.000 GiB is GPU-addressable at the macOS default
Storage	2.575 GB / 2.398 GiB (Q4_K_M)	2,574,807,904 bytes on disk; 4.130 GiB if you take Q8_0 instead
Software	macOS Sonoma 14+, llama.cpp ≥ b10153	Homebrew llama.cpp (currently 10330) or a source build

Why this chip: 546 GB/s against a model that streams twice

Token generation is bandwidth-bound, and the loop doubles what has to move. Parsing the tensor table out of the Q4_K_M GGUF header: the 22 decoder blocks are 1,865,048,064 bytes, the Q6_K LM head is 418,682,880, and the embedding table is a single-row gather. A generated token therefore moves 2 × 1,865,048,064 + 418,682,880 = 4,148,779,008 bytes ≈ 3.864 GiB, against 2.127 GiB for the same weights without the loop — 1.82× the traffic. This model occupies the memory of a 2.4 GB file and streams like a 4.1 GB one.

Which is exactly the axis this chip is strong on. A 48 GB M4 Max is unambiguously the 546 GB/s part, and it is worth pinning that down because the M4 Max ships in two bins with different bandwidth. Apple's own specification page lists them as "M4 Max with 14-core CPU and 32-core GPU (410GB/s memory bandwidth)" and "M4 Max with 16-core CPU and 40-core GPU (546GB/s memory bandwidth)", and the memory options are not the same across them: the 14-core part is sold at 36 GB and nothing else, while 48 GB, 64 GB and 128 GB are all 16-core-only (Apple, MacBook Pro 14-inch M4 specifications). So on this machine the capacity identifies the bin — and the inverse trap is worth knowing too: a 36 GB M4 Max is the slower bin, not the same chip with less memory. Quote a bandwidth figure only together with the bin it belongs to.

The practical consequence for this page is a negative one. 546 GB/s is roughly 1.37× the 400 GB/s that the 48 GB M3 Max and the M2 Max both run (Apple's spec page for the M3 Max), so a throughput figure measured on either of those would not transfer here, and neither would one measured here transfer to them. Since no measurement exists on any of the three, this page quotes no speed at all — see Results.

The ceiling is 36.000 GiB, not 48

Apple silicon has no dedicated video memory, and the capacity on the box is not what the GPU gets. Metal publishes a working-set ceiling in recommendedMaxWorkingSetSize, which llama.cpp-Metal, MLX and Ollama all treat as a hard cap. For a 48 GB machine that ceiling is 38,654,705,664 bytes = 36.000 GiB — precisely three-quarters of 48 GiB.

The figure is logged, not assumed. An owner describing his machine as a "MacBook Pro with 48 GiB memory" pasted a llama.cpp startup log to LM Studio's tracker containing "recommendedMaxWorkingSetSize = 38654.71 MB" (lmstudio-bug-tracker #271); an unrelated report on ollama #12012 prints the same value.

Check the units before converting, because the label will not do it for you. Current ggml divides this field by 1e6 and labels it MB (ggml-metal-device.m:946), and three-quarters of 48 GiB renders under that convention as exactly 38654.71. But a 2023-era build at tag b1180 printed the very same MB suffix while dividing by 1024² — so an old log's "MB" means MiB and a current one's means MB, and reading this one the wrong way would give 37.749 GiB, or 78.6% of the machine, a fraction of nothing in particular. What settles it without relying on the label is a line in the same log printed in unambiguous MiB: "using device Metal (Apple M4 Pro) - 36863 MiB free" — a 36,864 MiB pool, which is 36.000 GiB.

The caveat that comes with it. Every 48 GB log available is from an Apple M4 Pro, not an M4 or M3 Max. The ceiling looks like an OS policy keyed on installed memory rather than on GPU core count — each capacity we have logs for lands on an exact fraction of its own GiB size regardless of the chip — but that is our inference, not a statement from Apple, and it is the direction that would overpromise if wrong, since 36.000 GiB is more generous than the two-thirds share smaller Macs receive. The configuration this page recommends keeps 9.910 GiB in hand, so it tolerates a materially lower ceiling; the top of the ladder does not.

For reference in the other direction, the two-thirds regime applies at and below 32 GB (a 32 GB Mac addresses 21.333 GiB), while a 36 GB machine logs 28991.03 MB = 27.000 GiB — three-quarters again, and the number that sets this recipe's floor.

Runtime choice: llama.cpp with Metal

Why llama.cpp. Mainline merged this architecture on 2026-07-27 and registers it as nanbeige; Metal is the default backend on macOS; and you can pin a build number. It is also the runtime that exposes the KV-precision flags the lead configuration depends on.

The MLX shelf is crowded and mostly unloadable. A HuggingFace search returns 27 MLX-tagged repositories for this model — enumerated rather than sampled on 2026-08-10, 27 of 78 hits, the same count as a week earlier. Twenty-three cannot be loaded by stock mlx-lm at all: it resolves an architecture by importing mlx_lm.models.<model_type>, and among the 121 files in mlx_lm/models/ there is no nanbeige.py. HuggingFace's custom_code tag is a red herring here, because mlx-lm never reads auto_map. The other four ship their own MLX implementation and declare it via a model_file key in their config.json, which mlx-lm imports under trust_remote_code — three jishnuvenugopal/Nanbeige4.2-3B-mlx-{4,6,8}bit repos with nanbeige.py, and WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit with an independently written nanbeige_mlx.py. Those four do load, with --trust-remote-code. Preferring llama.cpp over them is a provenance judgement — reviewed upstream C++ you can pin, rather than one person's Python model definition self-pinned to mlx-lm internals — not a claim that MLX is unsuited to a looped transformer.

The arithmetic is the same on either path. That community port's make_cache returns num_loops * num_hidden_layers slots — 44, the same number llama.cpp, the HuggingFace reference and the Swift port allocate. The doubling is an architecture cost; only the flag names differ between runtimes.

Two model-card sections to skip. Its llama.cpp instructions clone the vendor's fork and configure the build with a CUDA backend flag; mainline superseded that fork and there is no CUDA on a Mac. And the org's -FP8 and -GPTQ-Int8 variants have no Apple execution path — no more than bitsandbytes, AWQ, Marlin or ExLlamaV2 do, none of which has a Metal kernel. The GGUF K-quants are the Apple equivalent.

Installation

1. Install llama.cpp with Metal

brew install llama.cpp
llama-cli --version

The Homebrew formula is at build 10330 today; anything from b10153 up carries the Nanbeige architecture. Metal needs no build flags on macOS — it is the default backend. From source instead:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build
cmake --build build --config Release -j

2. Fetch the GGUF

The run command below pulls and caches the repo itself, so this step is optional. To hold the file yourself:

pip install -U huggingface_hub

hf download owao/Nanbeige4.2-3B-GGUF Nanbeige4.2-3B-Q4_K_M.gguf \
  --local-dir ~/models/nanbeige4.2-3b

hf is the current HuggingFace CLI name — it was renamed from huggingface-cli in huggingface_hub v0.34.0, so on an older install the command is huggingface-cli download.

There is no first-party GGUF and no first-party MLX build: the Nanbeige org publishes ten repositories and all ten are safetensors-only, so every quant available is a community one. owao/Nanbeige4.2-3B-GGUF is the set to use — 13 rungs from Q2_K to Q8_0, re-uploaded in full on 2026-07-28 to pick up the vendor's chat-template fix. Check that date on any GGUF you substitute: a quant is a frozen copy of upstream at its upload timestamp, and this set was first published on release day, before the fix existed.

Running

llama-server \
  -hf owao/Nanbeige4.2-3B-GGUF:Q4_K_M \
  --host 127.0.0.1 --port 8080 \
  --ctx-size 262144 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  -ngl 99 \
  --temp 0.6 --top-p 0.95 --top-k 20

You get an OpenAI-compatible /v1/chat/completions endpoint on http://127.0.0.1:8080 and a browser UI at the same address. The cache is allocated at startup, so a context that does not fit fails in the first few seconds rather than mid-conversation.

The bill is 2.398 GiB of weights, 23.375 GiB of quantized cache and 0.317 GiB of reserved logits = 26.090 GiB against 36.000 GiB addressable — 9.910 GiB spare. The model card says "The model supports a context length of up to 262,144 tokens (256K).", and llama.cpp's --ctx-size is total context, prompt plus generation, so this is that whole window in one pool rather than a per-turn allowance.

That is 72% of the addressable pool, which would be an aggressive lead on a small Mac and is not one here: 26.090 GiB out of a 48 GB machine leaves roughly 20 GB outside the Metal ceiling for macOS and whatever else is open.

--cache-type-k and --cache-type-v are the long forms of -ctk / -ctv, and they must name the same type — see Troubleshooting. The sampler values are the card's own defaults for reasoning and chat; the card suggests --temp 1.0 for agentic and tool-use work, but read the tool-call entry under Troubleshooting before you take it.

If you would rather not quantize the cache

Halve the window and keep an exact cache:

llama-server -hf owao/Nanbeige4.2-3B-GGUF:Q4_K_M --host 127.0.0.1 --port 8080 \
  --ctx-size 131072 -ngl 99 --temp 0.6 --top-p 0.95 --top-k 20

24.715 GiB — 1.375 GiB cheaper than the lead, for half the context. Nobody has published what q8_0 costs this model in output quality, so which trade is right is a question for your own prompts. The combination that is simply unavailable at 48 GB is the full window with an f16 cache: 46.715 GiB against a 36.000 GiB pool, over by 10.715 GiB.

The memory budget — the loop tax

Where the parameters are. config.json gives 22 layers of hidden_size 3072, 48 query heads over 8 KV heads at head_dim 128, intermediate size 10752, an untied 166,144-token vocabulary and num_loops: 2. Summed, that is 4,169,800,704 parameters, 3,149,011,968 of them non-embedding — matching exactly the count HuggingFace reports for the safetensors, which settles the naming: the "3B" is the non-embedding figure. The census closing on one copy of the stack is the proof that the loop is free in weights, and the Q4_K_M file's 201 tensors (22 blocks × 9, plus embedding, output norm and LM head) confirm it on disk.

Where the memory goes. With 8 KV heads at 128 dimensions, a layer holds 2,048 K+V elements per token. load_arch_hparams() sets hparams.n_layer_all = n_layer_phys * n_loops = 44, and the allocator in src/llama-kv-cache.cpp reads that field directly — const uint32_t n_layer = hparams.n_layer_all; — making one K and one V tensor per slot. Per token:

44 × 2048 × 2 bytes = 180,224 bytes = 176.0 KiB at f16, versus 88.0 KiB for the same stack run once.

cache type	bytes/token	@32,768	@65,536	@131,072	@262,144
f16 (default)	180,224	5.500 GiB	11.000 GiB	22.000 GiB	44.000 GiB
q8_0	95,744	2.922 GiB	5.844 GiB	11.688 GiB	23.375 GiB
q4_0	50,688	1.547 GiB	3.094 GiB	6.188 GiB	12.375 GiB

A third term that is neither weights nor cache. The Metal compute buffer carries one reserved tensor that is unusually large on this model: the logits. llama.cpp sizes a worst-case graph once at startup, and src/llama-context.cpp reserves n_outputs_pp = std::min(n_tokens, cparams.n_outputs_max) rows, with n_tokens itself std::min(cparams.n_ctx, cparams.n_ubatch) and n_outputs_max defaulting to n_batch. At stock settings that is 512 rows regardless of --ctx-size, each an f32 distribution across the vocabulary:

512 rows × 166,144 vocab × 4 B = 340,262,912 B = 324.5 MiB = 0.317 GiB

It is worth its own column for two reasons. --ubatch-size moves it and --ctx-size does not — which makes -ub the cheapest lever when a configuration lands a couple of hundred mebibytes over. And it is only this large because the vocabulary is: at 32,000 tokens the same graph would reserve 62.5 MiB.

Weights plus cache plus logits, against the 36.000 GiB this Mac addresses:

configuration	weights	KV	logits	total	verdict
Q4_K_M + q4_0 @ 256K	2.398	12.375	0.317	15.090 GiB	Fits easily; coarsest cache
Q4_K_M + q8_0 @ 128K	2.398	11.688	0.317	14.402 GiB	Fits; half the window, half the cache
Q4_K_M + f16 @ 128K	2.398	22.000	0.317	24.715 GiB	Fits — the exact-cache option above
Q4_K_M + q8_0 @ 256K	2.398	23.375	0.317	26.090 GiB	Fits — this recipe
Q5_K_M + q8_0 @ 256K	2.782	23.375	0.317	26.474 GiB	Fits — headroom spent on weights
Q8_0 + q8_0 @ 256K	4.130	23.375	0.317	27.822 GiB	Fits, but costs bandwidth — see above
Q4_K_M + f16 @ 256K	2.398	44.000	0.317	46.715 GiB	Over by 10.715 GiB

Note what the bandwidth argument does to that table. The two rows that spend headroom on heavier weights, Q5_K_M and Q8_0, both fit — but on a machine whose binding resource is bytes moved per token, buying 1.7 GiB more weights costs traffic on every generated token and buys nothing this page can quantify. The headroom is better spent on context, which is where the lead configuration puts it.

How the 36 GB floor is settled rather than assumed. The addressable share is an exact fraction of capacity, not a range, so the comparison decides cleanly in both directions. A 36 GB Mac addresses exactly 27.000 GiB and clears the lead configuration by 0.910 GiB — before roughly 25 MiB of attention scratch, so about 0.886 GiB net, well outside rounding. A 32 GB Mac falls under the two-thirds regime at 21.333 GiB and is 4.757 GiB short. Neither is a margin call, which is why this recipe's floor is 36 rather than a hedge.

Two costs Metal does not impose. A quantized cache buys no dequantisation copy here. On CUDA the flash-attention path stages an additional f16 buffer whenever the K cache is not already f16, sized from ggml_nelements(K) in fattn-common.cuh, so there a quantized cache returns part of its saving in proportion to context. Metal's encoder has no counterpart: ggml_backend_metal_buffer_type_get_alloc_size appends exactly three regions to a flash-attention output, none of which copies K or V. Sized from ggml-metal-ops.cpp for 48 heads and a 128-wide value head, the largest is a per-workgroup result buffer of 4 × 32 × 48 × 32 × 130 bytes = 24.375 MiB with no context term; the only context-linear region is a block-skip bitmap of one byte per 64 cache positions per 8 query rows, which reaches 256 KiB at 262,144 tokens. The q8_0 row above is therefore the entire cache cost on this machine, which is what makes 26.090 GiB a flat figure instead of one that grows.

The attention mask is outside the Metal ceiling too. llm_graph_input_attn_kv::set_input writes it straight to tensor->data and so asserts ggml_backend_buffer_is_host(self_kq_mask->buffer) (llama-graph.cpp:450); both Metal buffer types return an unconditional false from their is_host callback (ggml-metal.cpp:275), so the scheduler puts the mask on the CPU backend. On unified memory that is not "elsewhere in the machine" — one pool of DRAM serves both, and those bytes still come out of your 48 GB. It only means they are not charged against the 36.000 GiB ceiling this page measures against.

The loop was chosen, not inherited. The team built and rejected a KV-sharing variant — "we did try sharing the KV cache across loop passes, but it noticeably hurt performance, so we kept the full cache in Nanbeige4.2." (discussion #18, Nanbeige team member leran1995). Sharing would have halved this table; they kept model quality instead. No flag reverses that — only cache quantization moves the number.

Results

• Speed: omitted. There is no throughput measurement for this model on the llama.cpp-Metal path on any Apple chip, and none on an M4 Max on any runtime — not the model card, not the technical report (neither prints a tokens-per-second figure anywhere), not the 28 discussions on the canonical repo, not the three on the GGUF repo, and not a targeted web search naming this chip. One Apple measurement does exist, on a different runtime and a very different machine, and it is worth naming exactly rather than pretending it is absent: the author of the community MLX port ran a 30-case agentic eval with greedy decoding on hardware he records as "Apple M1 Pro, 10-core, 16 GB unified memory" and reports 35.1 tok/s aggregate decode at MLX 4-bit, with the per-case results committed alongside (nanbeige-mlx-eval). Four things bound it: single community source, the MLX runtime rather than this recipe's GGUF, short tool-use prompts rather than a long window, and an entry-tier chip several generations and a large memory-bandwidth gap below this one. It is a floor this machine should clear by a wide and unquantified margin — not an estimate of it — so no figure is projected onto the M4 Max here. Measure it and contribute the numbers, and /check/nanbeige4-2-3b/m4-max stops being empty.
• Unified memory usage: 26.090 GiB derived for the configuration above — 2,574,807,904 bytes of Q4_K_M weights, 25,098,715,136 bytes of q8_0 KV at 262,144 tokens, and 340,262,912 bytes of reserved logits — against 36.000 GiB addressable, leaving 9.910 GiB. This is arithmetic over measured file bytes, the runtime's 44-layer allocation rule and its worst-case graph reservation; it is not a measured peak, and one term sits on top of it: the flash-attention scratch sized above at roughly 25 MiB, near-flat in context. See /check/nanbeige4-2-3b/m4-max.
• Quality notes: the card's benchmark tables were produced in thinking mode with preserve_thinking=true, i.e. long reasoning traces — so the cache table above is not a worst case you can discount, and it is much of the argument for taking the full window on a machine that can hold it. The card recommends preserve_thinking=False for general chat and True for multi-turn tool use, office tasks and code-agent workflows. No quality delta between f16, q8_0 and q4_0 caches is claimed anywhere on this page, because none has been published for this model.

For the full benchmark data, see /check/nanbeige4-2-3b/m4-max.

Troubleshooting

error loading model: unknown model architecture: 'nanbeige'

Your build predates 2026-07-27. The architecture landed in mainline via PR #25994 and the first release carrying it is b10153 — src/models/nanbeige.cpp returns 404 at b10152 and 200 at b10153 — so brew upgrade llama.cpp fixes it. LM Studio prints the same message when its bundled backend is older — the card's instruction to hand-copy a fork's binaries into the backend directory predates the merge and can be ignored once your backend is current. Stock Ollama has no entry for this model at all: ollama.com/library/nanbeige, /nanbeige4.2, /nanbeige4-2 and /nanbeige4.2-3b all 404, and the nanbeige namespace assumed by the card's ollama run command does not exist either.

Keep --cache-type-k and --cache-type-v the same type

On Metal this is a hard requirement, not a preference. ggml_metal_device_supports_op refuses the fused-attention op when the K and V types differ — ggml-metal-device.m:1291 is a bare if (op->src[1]->type != op->src[2]->type) { return false; } — so a mismatched pair silently moves your attention off the GPU rather than raising an error. The same function whitelists the usable cache types: f32, f16, q8_0, q4_0, q4_1, q5_0, q5_1, plus bf16 on devices that support it; anything else, iq4_nl included, is refused identically. This model's head_dim of 128 is on the supported head-size list, so the type pairing is the only thing to get right.

A quantized V cache additionally requires the fused-attention path, which llama.cpp enables for you at the default -fa auto. That is not FlashAttention. -fa selects ggml's own fused-attention kernel, which has a Metal implementation compiled into the binary Homebrew just installed; the CUDA flash-attn package has no Apple build, is not a dependency of anything on this page, and must not be installed.

--ctx-size 0 asks for 44 GiB and dies immediately

0 is not "use the default". common/arg.cpp sets params.fit_params_min_ctx = UINT32_MAX when you pass it, disabling automatic context reduction and requesting the model's full 262,144 tokens at the default f16 cache — 44.000 GiB, far past this pool. Name both the context and the cache types explicitly, as the run command does.

Related: llama-server's slots do not divide your cache. It sets n_parallel = 4 and kv_unified = true in the same branch, and a unified cache gives n_ctx_seq = n_ctx, so --ctx-size 262144 buys one conversation the entire window from a single shared pool. The 23.375 GiB above is the whole cost, not a quarter of it.

ValueError: Model type nanbeige not supported. from mlx-lm

Expected on 23 of the 27 MLX builds, for the reason given in the runtime-choice section: stock mlx-lm has no nanbeige architecture. Nothing is wrong with the repo you picked — including the one under mlx-community, which is in the failing group. Either take the GGUF path, or switch to one of the four repos that ship their own model_file implementation and pass --trust-remote-code.

One of those four is a trap. WaveCut/Nanbeige4.2-3B-heretic-MLX-DWQ-4bit loads, but it is an abliterated derivative rather than a quantization of the base model, so "it was the one that worked" is the worst possible reason to settle on it. It is not a like-for-like substitute for anything else here.

Tool calls arrive as plain text in message.content

Fixed in mainline as of release tag b10227 — check your build first. This was llama.cpp's parser, not the model: <tool_call> is sometimes emitted with a trailing space where the auto-derived marker expected a newline, the match failed, and the call was returned as ordinary content. PR #26324 put the pre-fix rate at roughly a quarter of calls, but that patch was never merged — a maintainer noted that blanket whitespace trimming had been rolled back once before for hurting Qwen-family output. The repair came from a different direction: PR #26252, a specialized parser merged on 2026-08-02 and released as b10227. #26324's author closed his own PR on 2026-08-10 with "Tried b10335 (#26252 was merged in b10227) and nanbeige tool calls work 100% of the times, thank you".

The practical consequence for this machine is one number: b10227 is the floor for tool calling, above the b10153 floor this page requires for the architecture itself. The Homebrew formula quoted under Installation is build 10330 and therefore carries both. Nothing needs working around, the card's --temp 1.0 for agentic work can be taken as written, and if calls still arrive as text the answer is brew upgrade llama.cpp.

Is a quantized cache slow at 262,144 tokens?

A report opened on 2026-08-10 argues that implementations with a quantized KV cache read it as f16, quadrupling DRAM traffic and making long-context inference very slow. Treat it as unverified: no maintainer has responded, it names no backend, and the same reporter's other posts in that repo's discussions describe an 8 GB discrete NVIDIA card (discussion #18) — the backend whose flash-attention path genuinely does stage an f16 copy, per the CUDA source cited above, and which the Metal source has no equivalent of. It is also a throughput claim, not a memory one, so nothing in this page's budget moves either way. On a chip whose whole argument is bandwidth this is the measurement most worth having: if you can run it, send it in.

Memory pressure or an allocation failure at startup

If you are a couple of hundred mebibytes short, try --ubatch-size 256 first — it halves the logits reservation to 162.25 MiB, costs prompt-processing throughput only, and touches neither the window nor decode speed. After that, step --ctx-size down a rung or switch to --cache-type-k q4_0 --cache-type-v q4_0, before considering sudo sysctl iogpu.wired_limit_mb=<MB>. The raise is rarely the right move on this model, because cache quantization recovers more for less risk; if you do use it, leave 8–16 GB for macOS, watch Activity Monitor's memory-pressure gauge, and remember it is temporary and resets on reboot, with 0 restoring the default.

Do not install the Python path just to try it

The card's HuggingFace quickstart needs trust_remote_code=True and a narrow dependency set; a community write-up only got it working after pinning Python 3.11, PyTorch 2.8.0, Transformers 4.42.4 and more, having first produced gibberish on a mismatched configuration (discussion #15). It also pulls CUDA-only attention packages with no Apple equivalent. The GGUF route above needs none of them.

Nothing else widely reported. Problems, or a throughput number for this chip, go to the submission form.