Apodex 1.1 mini on Apple M2 Max: 6-bit MLX at 131K, and why 262K is a different tier

What You'll Build

An OpenAI-compatible agent endpoint running Apodex 1.1 mini — Apodex AI's reasoning-first agent model, a 35.95B-parameter Mixture-of-Experts post-trained from Qwen/Qwen3.5-35B-A3B — on a 64 GB M2 Max, served by mlx_lm.server at 6-bit, documented to a 131,072-token conversation, rather than the 4-bit that the 48 GB M4 Max page documents.

The quant choice is the whole point of this page, and on this machine it is not a question of what fits in the weights budget — every published MLX tier does. It is a question of what fits while prefilling a long prompt, because that transient cost is large, it grows with context, and it turns out to be almost independent of which quant you chose. That makes the tier and the window a single joint decision rather than two separate ones, and it is why this page leads 6-bit at 131K instead of 6-bit at the model's full 262,144.

Hardware data: Apple M2 Max (64 GB unified memory, 400 GB/s) · resident working set 28.794 GiB at 131,072 tokens, derived peak ~40 GB during prefill, against a 48.000 GiB (51.54 GB) addressable pool · See benchmark data

⚠️ The resident budget clears the full 262,144-token window; the peak during a long prefill is a separate, larger number, and it is what decides your ceiling. Every weights-plus-KV figure on this page is a resident budget. The measurement this page's transient term rests on is a comment on mlx-lm #1480 by galashko, 2026-08-13, and it is worth reading before you pick a window: on an Apple M5 Max with 128 GB, mlx 0.32.0 and mlx-lm 0.31.3, running mlx-community/Qwen3.6-35B-A3B-4bit — architecturally identical to this model, verified field by field — "176k and 192k both complete here", with peak memory climbing "1.4 GB per 16k tokens" and, in their words, "the curve has no knee". Subtracting that repo's own resident bytes — and reading the rung labels off the harness's raw prompt_tokens rather than off the comment's rounded table — leaves a transient term of about 0.68 GB per 10,000 tokens. Carrying it onto this page's budget puts 6-bit at 131,072 near 40.4 GB, comfortably inside the pool, and at the full 262,144 at 52.0 GB against a 51.54 GB pool — over the line on the central estimate, with a band of 50.1–53.3 that still contains it. That is why this page documents 131K and treats the full window as probably-just-over rather than promised. The whole derivation, including what it assumes — and a second comment on that same thread, posted 2026-08-30, whose OS-level measurement of the same term is an order of magnitude larger in a different runtime, which makes the ceiling a wide bracket rather than a number — is in the peak-memory section.

ℹ️ Unified memory is not VRAM. Apple Silicon has no dedicated VRAM. Our catalogue stores full unified memory in the GPU row, but Metal's recommendedMaxWorkingSetSize caps what the GPU may actually address: exactly 3/4 on machines of 36 GB and above, so a 64 GB Mac addresses 48.000 GiB, not 64. That figure is measured rather than assumed, and two primary logs from two eras of ggml's printing agree on the same byte value: llama.cpp #9112 on a 64 GB M2 Max logs recommendedMaxWorkingSetSize = 51539.61 MB (printed with /1e6), and llama-cpp-python #948 on a 64 GB M1 Max logs 49152.00 MiB (printed with /1024/1024). Both are 51,539,607,552 B = 48.000 GiB = 0.75 × 64 GiB. Every figure below is in GiB against that pool, never against the marketing 64.

ℹ️ This is a text-only recipe, and that is a property of the artifact, not a hedge. mlx-lm's loader for this architecture discards the checkpoint's vision weights before the model is built — qwen3_5_moe.py skips any key matching if key.startswith("vision_tower") or key.startswith("model.visual"): — and it drops the multi-token-prediction head one layer down with weights = {k: v for k, v in weights.items() if "mtp." not in k}. Read out of the safetensors headers directly, all five MLX builds of this model carry 1,757 tensors, zero vision keys and zero mtp. keys, and their config.json has no vision_config block at all where the first-party checkpoint has one. The publisher says so too: abenzerps's card reads "This is a text-only MLX release. Image input is not supported." Apodex's own card is pipeline_tag: text-generation and the words vision, visual, multimodal and mmproj appear zero times in it, against 24 occurrences of agent as a control.

ℹ️ An MoE keeps every expert resident; only the reads are sparse. This model activates 8 of 256 experts per token, which is a routing property, not a memory one. All 256 stay in memory, so the footprint is the whole quantised file. What the routing does buy is bytes per step, and that is the axis this page's quant decision turns on.

Requirements

Which M2 Max this is, and what that does not tell you. Apple sells the M2 Max with a 30-core GPU as standard, configurable to a 12-core-CPU / 38-core-GPU part, and per Apple's MacBook Pro (14-inch, 2023) specifications the memory options are 32 GB standard, configurable to 64 GB or — on the 38-core part only — 96 GB. So 64 GB does not identify the GPU bin here, unlike the 48 GB M4 Max case where it does. It does not need to: the same page gives 400GB/s memory bandwidth for the M2 Max as a single figure for the chip, without the per-bin split it draws elsewhere for CPU and GPU cores — which is the evidence that both M2 Max GPU bins run the same bandwidth, and it is the opposite situation from the M4 Max, where the two bins genuinely differ (410 vs 546 GB/s) and the memory size is what tells them apart. Nothing on this page depends on which bin you have, because nothing on this page quotes a throughput number; a future page that does will have to say which bin it was measured on.

The memory budget (derived, not measured)

Nothing here was observed on a machine. Every input is a byte count read out of a safetensors header or a field read out of the model's own config.json, and the arithmetic is shown so you can redo it.

The architecture is hybrid: of its 40 decoder layers only some carry a context-scaling KV cache. config.json declares full_attention_interval: 4 and a layer_types array that counts out to 30 linear_attention and 10 full_attention entries; mlx-lm derives the identical split independently with self.is_linear = (layer_idx + 1) % args.full_attention_interval != 0 and allocates from it — [ArraysCache(size=2) if l.is_linear else KVCache() for l in self.layers]. Only 10 of 40 layers hold a cache that grows with context, which is the whole reason a 262K window is a laptop conversation at all.

The lead build, at the 131,072-token window this page documents, one sequence:

At the model's full 262,144 tokens the same four addends give 33,601,485,056 B = 31.294 GiB resident, 16.706 GiB inside the pool. Resident is not the binding number, though — the next section is.

Per-token KV is 2 (K + V) × 2 KV heads × 256 head dim × 2 bytes = 2,048 B per full-attention layer, which is 20 KiB/token across the 10 of them: 32,768 tokens costs 0.625 GiB, 131,072 costs 2.500 GiB, and the full 262,144 costs 5.000 GiB. The 2-byte element is the model's own bfloat16 (config.json → text_config.dtype), because mlx_lm.server exposes no KV-quantisation flag — --kv-bits ("Number of bits for KV cache quantization. Defaults to no quantization.") exists only on mlx_lm.generate.

The recurrent half does not grow with context. Each linear layer holds a Gated-DeltaNet state allocated as state = mx.zeros((B, Hv, Dv, Dk), dtype=mx.float32) — 32 value heads × 128 × 128 at float32 = 2.000 MiB — plus a (kernel−1) × conv_dim = 3 × 8192 bfloat16 conv state of 48 KiB. Thirty layers of that is 0.060 GiB, constant, whether you run 1K or 262K.

A note on which bytes these are. The weights row is the summed tensor payload read from the safetensors headers, not the on-disk file total. The two differ by the JSON headers — 28,168,386,816 B resident against 28,168,623,119 B on disk, a gap of 236,303 B — which is why the Storage row above says 28.17 GB and this one says 26.234 GiB, and why a figure here can differ from a sibling page's in the third decimal. Disk size and residency are different numbers and this page does not swap them.

What this budget does not count. This is the resident set: weights, KV cache, recurrent state. It does not include the transient workspace a prefill allocates, which scales with --prefill-step-size ("Step size for prefill processing (default: 2048)") and with prompt length. On this architecture that term is not a rounding error — it is the largest single uncertainty on the page, and its public evidence is two comments on one issue thread, which disagree by more than an order of magnitude.

The peak during prefill, and why it changes the answer

The published peak-memory data for this architecture is two comments on mlx-lm #1480. The ladder below is the first of them — galashko, 2026-08-13. It is one community datapoint, on a different checkpoint and a different chip, and everything below inherits those limits. It is also the only one taken in the regime this recipe runs — one request at a time, a fresh process per rung — so here it is in full rather than summarised away. The second, lueghausen on 2026-08-30, measures a transient more than an order of magnitude larger, in a different runtime on a different quantisation. The end of this section sets the two side by side; the result is a wide bracket rather than a resolved number, and this page's lead sits at the low end of it because that is the end whose configuration this recipe runs.

The setup, from the harness repo the comment links: Apple M5 Max (40 GPU cores, 128 GB), macOS 26.6.1, mlx 0.32.0, mlx-lm 0.31.3, mlx-community/Qwen3.6-35B-A3B-4bit, each rung in its own process, greedy, 8 generated tokens, median of two full runs — and the two published runs report the identical peak_gb at every rung, so there is no spread to weigh. The headline is "176k and 192k both complete here", which matters because the issue it replies to is a 128 GB machine failing at 176k. That is a real disagreement between two 128 GB machines, but it is not a controlled one: the failing setup ran mlx 0.31.2 through mlx_lm.server with --prompt-cache-size 8 --prompt-cache-bytes 32GB, the passing one ran mlx 0.32.0 as a fresh process per rung. Version, code path and cache configuration all differ, so the pair establishes that the wall is not a fixed function of capacity — not what does set it.

Read the rung labels off the raw data, not off the table. The comment tabulates "8k … 192k"; the harness's results/mlx-q4.jsonl records prompt_tokens of 8,022 / 32,022 / 64,022 / 100,022 / 130,022 / 160,022 / 176,022 / 192,022 — decimal thousands plus 22 tokens of chat template, not multiples of 1,024. It matters: reading them as 8,192 … 196,608 flattens the slope by about 3% and moves this page's 262,144 verdict across the pool boundary.

Only the second column is measured. The third is this page's own derivation from that repo's byte count — mlx-community/Qwen3.6-35B-A3B-4bit is 20,402,204,271 B of safetensors against this model's 19,508,787,456 B, so subtracting the wrong one inflates the transient by 0.89 GB — plus the KV and recurrent formulas above, which carry across unchanged because their config.json is identical to ours field for field (40 layers, 30 linear / 10 full, full_attention_interval: 4, 2 KV heads × 256, bfloat16, group_size: 64 affine with 80 overrides). So the fourth column is a measurement minus a derivation, not a difference of two measurements.

The ladder was measured at the step size this recipe runs. step_mlx.py calls stream_generate with no prefill_step_size and no draft model, so it takes generate_step's default of 2048 — the same value mlx_lm.server --prefill-step-size defaults to. (The 512 default in the file belongs to speculative_generate_step, which needs a draft model and is unreachable on this architecture anyway.) So the transient below is the transient at this page's own configuration, not at some other chunk size — which is what makes it the right comparison for this recipe's own configuration, and also why lowering that flag is not the escape hatch it looks like. See the reconciliation at the end of this section.

The unit is settled by source, not by inference. The harness writes peak_gb from mlx-lm's peak_memory, which is mx.get_peak_memory() / 1e9 — decimal GB of MLX's own Metal allocator peak. That is the right quantity to compare against recommendedMaxWorkingSetSize, and it is why every figure in this section is GB rather than GiB.

Two properties fall out:

• The transient grows at about 0.68 GB per 10,000 tokens (per-rung spread 0.61–0.73; least squares 6.8005e-5 GB/token, intercept 0.522). The comment's headline "1.4 GB per 16k tokens" is the slope of the total curve and it reconciles exactly: 1.451 GB per 16,384 tokens measured, minus the 0.336 GB per 16,384 the KV cache contributes at 20,480 B/token, leaves 1.116, against the least-squares transient slope of 1.114 in that unit — the 0.002 between them is an endpoint slope versus a fitted one, not a disagreement. Both figures are correct about different quantities, which is worth stating because the difference between them is the KV term and not an error.
• The transient appears to be quant-independent, which is what licenses carrying a 4-bit ladder onto 6-bit. Computed at the same rung, the transient implied by the 4-bit ladder is 13.60 GB and the one implied by the reported 8-bit peak is 13.58 GB — agreeing to 0.02 GB. This is the weakest leg of the argument and it should be treated as such: the harness repo publishes only 4-bit ladders, so that 8-bit number has no raw file behind it, and the author explicitly flags that comparison as coming from a separate run on a different corpus, "so not directly comparable to the tables above", and reports it only as a direction. One non-comparable figure agreeing to 0.02 GB is a coincidence-sized sample.

Carrying that transient onto this page's own measured bytes gives the peak for each tier. The band is the per-rung slope spread, not a confidence interval:

The middle column is galashko's own top rung, which makes it a round-trip check on the whole method: this model at 4-bit comes out at 37.1 GB there, and their checkpoint carries 0.893 GB more weight, so the same arithmetic predicts 37.98 GB for their run — against the 38.0 GB they measured.

Against the 51.54 GB pool. Three readings, in decreasing confidence:

1. 6-bit at 131,072 clears under every slope in the band, at 40.4 GB with about 11 GB to spare. That is the lead.
2. 8-bit does not reach the full window on this machine — 58.7–62.0 GB against 51.54 — and its ceiling lands near 151K–159K. Resident bytes alone say the opposite (39.359 GiB sits comfortably inside 48.000), which is exactly the reading this section exists to correct.
3. 6-bit at the full 262,144 is over the line on the central estimate: 52.0 GB against 51.54, with a band of 50.1–53.3 that still contains the boundary. Its ceiling is roughly 243K–257K depending on which slope you take. This page will not call that fitting; the honest reading is "probably just over, and nobody has measured it".

What this is not. It is an extrapolation from one reporter, on a different checkpoint, a different chip generation and a newer mlx, and past 192,022 tokens it is beyond the measured range entirely. The harness README adds a caveat that cuts directly at transferring it: "Content affects the numbers, not just length." — the same ladder on a different corpus measured roughly a third slower at 8k, so token composition moves these curves too. Treat the tiers' ordering as solid and the absolute figures as indicative. And if you read the comment yourself, note that its closing 65 GB extrapolation for the full 262k context is about 50% above what its own table supports — carrying its stated 1.4 GB/16k slope the four rungs past its top rung gives about 43.6 GB. This page uses that comment's measurements and not its extrapolation.

A second comment on that thread measures +35 GiB, and this page cannot reconcile it with the ladder above. On 2026-08-30 lueghausen posted to the same issue: an M5 Max with 128 GB running Qwen3.6-35B-A3B in nvfp4, weights ~22 GiB, "peak during prefill" 56.7 GiB — a transient of +35 GiB, more than an order of magnitude above every rung in the table. You should know what that is before you plan a window, and the honest answer is not that one of the two reports is wrong.

It is not a counter artifact. The figure is "Measured with footprint --pid" — the operating system's own accounting of the process, which no allocator counter can inflate — and in the companion report ml-explore/mlx #3896 the same reporter logs footprint --pid at "93.2 / 93.6 / 94.7 / 99.2 GB" across four separate interventions on that 128 GB machine, while ollama ps showed 20 GB and ps -o rss showed 15 GB at the same moment. A kernel panic and a hard freeze followed. The memory came back afterwards — "memory is released between requests, so it is transient, not accumulation" — so it is the same kind of term the ladder measures, at a very different size. Any monitoring you build on ps will not see it.

Three things separate the two reports. All three are real, and nothing published today ranks them. Quantisation: those rows are nvfp4; every tier on this page is MLX mode: affine, a different kernel path with a different workspace. Runtime: that session ran through Ollama 0.33.1/0.33.2, not mlx_lm.server. Session shape: "multi-turn agentic coding sessions with tool calls" with prompts "growing to 90–115k tokens" over hours, against galashko's single request in a fresh process per rung. Nothing isolates which of the three carries the difference, or whether it is all three.

And the ladder's own instrument reads low, in the same direction. galashko's peak_gb comes from mx.get_peak_memory(), a high-water mark of the allocator's active bytes only — peak_memory_ = std::max(peak_memory_, active_memory_); at mlx/backend/metal/allocator.cpp :167 and :215, v0.32.0, the only two lines in that file that touch it — so it excludes buffers freed into the pool and still resident. #3896 measures that gap at 60.06 GB of pool against a 1.00 GB reported peak on a churning synthetic, and the maintainer closing it, zcbenz, calls the counter "mostly useful for measuring the memory consumption when ineferencing/training a single model", recommending get_active_memory() + get_cache_memory() instead. One request in a fresh process is the case that counter is best at, so the gap there should be small — but it is one-directional. Treat the ladder as a floor on the transient, not a ceiling.

So the honest reading is a bracket, and it is wide enough to matter. At the low end, galashko's slope puts 6-bit at 131,072 tokens near 40.4 GB against this machine's 51.54 GB pool. At the high end, lueghausen's +35 GiB on top of this page's own weights puts the same install at 61.2 GiB against a 48.000 GiB pool — it would not run at all, and neither would 4-bit, at 18.169 + 35 = 53.2 GiB. Nothing published sits between those two ends. This page leads the low end for one reason and states it rather than dressing it as margin: the configuration documented here is the one galashko measured — affine weights, mlx_lm.server, one request at a time, a two-entry prompt cache. Step outside it — Ollama, an nvfp4 build, or a long multi-turn agentic session — and you are in the half of the bracket nobody has measured on Apple hardware, where the one report that exists ended in a kernel panic on a machine with twice this one's memory. If you intend to run this as a 24/7 agent rather than a single-request endpoint, watch footprint --pid from the start and treat 64 GB as unproven for that use.

If you hit a prefill OOM, shorten the prompt — that is the lever the ladder actually measures, at about 0.68 GB per 10,000 tokens. Lowering --prefill-step-size is cheap to try and costs only prefill throughput, but do not plan a window around it: the second #1480 comment ran a fixed 2,048-token chunk and spiked anyway, reporting that "chunking at 2048 does not bound the transient allocation", and its llama.cpp cross-check on the same machine moved peak memory by only about 1.3 GiB across an eightfold change in batch width. Measure with footprint --pid: it is the instrument that agreed with the machine in both reports, on a box where ps -o rss was reading 15 GB against its 93 GB.

Why 6-bit, and not 4-bit or 8-bit

Every published MLX tier is resident-affordable here, which is why the resident table settles nothing:

The last column is the one that decides, and it comes from the peak model above, not from these resident figures. So the tier question is really "what should you spend the room on, at the window you actually work at". Three costs bear on it.

Cost one — bytes read per decode step. Token generation on this hardware is bound by memory bandwidth, and this is an MoE, so a step does not read the whole file: it reads every non-expert tensor plus the 8 routed experts of 256 that the router selects per layer. That gather lives one import away from the two files named above: qwen3_5.py pulls in Qwen3NextSparseMoeBlock from qwen3_next.py, where line 320 builds self.switch_mlp = SwitchGLU(dim, intermediate_size, num_experts) and line 343 calls it as y = self.switch_mlp(x, inds) with the top-k indices. Splitting the census by tensor name — routed experts are 92.9% of the weight bytes at every tier — gives a lower bound on the per-step read:

The ratio is the load-bearing column and it does not depend on the gather assumption: the whole-file ratios are 1.00 / 1.22 / 1.44 / 1.89, i.e. the same numbers, because the quantisation is applied almost uniformly across the checkpoint. Whether the runtime reads only the routed experts or all of them, 6-bit costs about 1.44× the per-step bandwidth of 4-bit and 8-bit costs about 1.88×.

Cost two — how many conversations the pool still holds. A distinct prompt cache at this page's 131,072-token window is 2.560 GiB (KV + recurrent + conv). Against a 48.000 GiB pool, after the weights, resident-only:

Read that as a steady-state ceiling, not a working number: a prefill in flight adds its own ~10 GB on top at this window, so the practical figure is several lower. mlx_lm.server defaults --prompt-cache-size to 10 with no byte cap, which at 6-bit is above what the pool holds even before the prefill term — which is why the run command below sets it explicitly.

Cost three — the prefill ceiling, from the peak model. This is the one that ranks the tiers differently from the other two: 4-bit and 5-bit reach the model's full window, 6-bit reaches somewhere around 243K–257K and falls just short of the full window, and 8-bit stops near 151K–159K.

The call. 6-bit at 131,072 leads. It buys two effective bits per weight over the 4-bit for 1.44× the per-step read, clears the peak model under every slope in the band with ~11 GB spare, and still holds eight caches. Its cost is the window: if your work genuinely needs the full 262,144 tokens, drop to 4-bit or 5-bit, which reach it with margin (43.3 and 47.6 GB against 51.54), rather than pushing 6-bit onto a boundary nobody has measured. 8-bit is documented rather than led — 1.88× the per-step read, a ~150K–160K ceiling, and nobody has measured what the extra two bits buy on this model, so it is a real cost against an unquantified benefit. 4-bit also stays the right build if you want maximum throughput or the strongest provenance (see below); it is what the 48 GB M4 Max page leads with — and on the tier choice that page and this one agree.

Where this page does contradict its 48 GB sibling, and it is deliberate. That page says the transient scales with --prefill-step-size "rather than with total context", and that on a 48 GB Mac "the 12.771 GiB of headroom absorbs that comfortably". The ladder above overturns both: the transient is the term that grows fastest with context — about 0.68 GB per 10,000 tokens, 3.3× the KV cache's 0.20 — and 12.771 GiB of headroom does not absorb it at a long window. That page's 4-bit lead survives the correction, because 4-bit is the tier with the most room; its reasoning about the prefill term does not. This page moved its own lead from 262K to 131K and its own floor from 48 GB to 64 GB on exactly that arithmetic, so it would be incoherent to apply the correction here and exempt the sibling.

Why 64 GB is this page's floor, and why it moved. The install documented here — the 6-bit build at a 131,072-token window — is 28.794 GiB resident, which a 48 GB Mac's 36.000 GiB pool would hold comfortably. Under the peak model it does not: 6-bit at 131,072 peaks near 40.4 GB against that machine's 38.65 GB pool, and a 48 GB Mac's 6-bit ceiling lands around 105K–112K. So min_vram_gb: 64 is a filter floor naming the smallest machine on which this install runs at this window — set from the peak rather than from the resident set, which is the whole lesson of the section above. On resident bytes alone this page would have said 48 and surfaced itself to machines that would OOM during a long prefill. Read that floor as conditional on the regime, not as headroom: it follows from the ladder, which is the low end of the bracket in the section above, and the high end of that bracket does not fit this machine at any tier. 64 GB is the smallest machine on which this install is documented to run at this window in the configuration written here — one request at a time, on mlx_lm.server, on affine weights. It is not a guarantee for a long agentic session, which nobody has measured on Apple at this size. If you are on 36 or 48 GB, the M4 Max page's 4-bit path is the one written for you.

Installation

1. Install mlx-lm

The architecture is implemented in stock mlx-lm: mlx_lm/models/qwen3_5_moe.py is present at tag v0.31.3, which is the current PyPI release (uploaded 2026-04-22). No --trust-remote-code, no git checkout, no third-party model file:

python3 -m venv .venv && source .venv/bin/activate
pip install -U "mlx-lm>=0.31.3"

There is nothing CUDA-shaped to install and nothing to configure for the GPU — MLX targets Metal directly. Skip pip install flash-attn, skip any cu12x wheel index, and ignore the first-party NVFP4 / GPTQ-Int4 / FP8 builds on the Hub: those are CUDA-native formats (NVFP4 is a Blackwell microscaling format, GPTQ kernels are CUDA-only, FP8 tensor cores are Hopper/Blackwell) and Apple Silicon has none of that hardware.

2. Download the 6-bit MLX weights

pip install -U huggingface_hub

hf download nicolasembleton/Apodex-1.1-mini-MLX-6bit --local-dir ./apodex-1.1-mini-mlx-6bit

mlx_lm.server --model nicolasembleton/Apodex-1.1-mini-MLX-6bit also resolves the repo id directly and downloads into the Hugging Face cache; the explicit hf download above is only for controlling where ~29 GB lands.

Provenance, since every MLX build of this model is a single-author community conversion. There is no mlx-community build to prefer. Enumerating the 76 repos the Hub returns for a search on Apodex and filtering on the mlx tag gives 40 hits, of which exactly five declare base_model: apodex/Apodex-1.1-mini — the four nicolasembleton tiers and abenzerps/Apodex-1.1-mini-MLX; the other 35 are builds of the earlier Apodex 1.0 family. Enumerated 2026-08-30, keyed on base_model rather than on the repo name, so a differently-named build would still have been caught.

Two checks make the single-author build safe to lead with, and both are cheap to repeat:

• The author's pipeline is reproducible. abenzerps/Apodex-1.1-mini-MLX and nicolasembleton/Apodex-1.1-mini-MLX-4bit are not merely the same size — all four shards have identical SHA-256 hashes (48ee541f…, 399ab1dd…, 11785640…, 7553a127…), confirmed against the SHA256SUMS file abenzerps publishes. Two people ran mlx_lm.convert at the same version and got the same bytes, which is what a deterministic affine quantiser should do and is direct evidence about the process that produced the 6-bit build too.
• The chat template did not drift. chat_template.jinja in the 6-bit repo is SHA-256 identical to the one in apodex/Apodex-1.1-mini (d139c773f9995a01434c0266ab2bd8d9859de30e56a27128c63d3439c57ddec0, 8,831 B). Redistributor template drift is a real failure mode for tool-calling models; this build does not have it.

All four nicolasembleton tiers declare group_size: 64, mode: affine with 80 per-tensor 8-bit overrides — the 40 layers' mlp.gate and mlp.shared_expert_gate routers, which is what abenzerps's card describes as "The MoE router and shared-expert gate weights remain at 8-bit". The 6-bit card reports 6.502 effective bits per weight against the 4-bit's 4.503.

Running

1. Start the server

mlx_lm.server \
  --model ./apodex-1.1-mini-mlx-6bit \
  --host 127.0.0.1 --port 8080 \
  --temp 1.0 --top-p 0.95 \
  --max-tokens 32768 \
  --prompt-cache-size 2

Every one of those flags is correcting a default that will otherwise bite you:

• --temp 1.0 --top-p 0.95. Apodex's card recommends temperature: 1.0, top_p: 0.95, repetition_penalty: 1.05, max_tokens: 32768 for agentic work. The server's own default is "Default sampling temperature (default: 0.0)" — greedy — with --top-p at 1.0. You have to pass them.
• --max-tokens 32768. The server default is 512. This is a reasoning model that opens a turn inside a <think> block; a 512-token ceiling truncates mid-thought and returns something that reads as a wrong answer rather than as a truncation.
• --prompt-cache-size 2. The default is 10 — "Maximum number of distinct KV caches to hold in the prompt cache" — and --prompt-cache-bytes, the only byte cap ("Maximum size in bytes of the KV caches"), is unset by default, so nothing trims by size unless you ask. Ten caches at this page's 131,072-token window is 25.6 GiB of cache on top of 26.234 GiB of weights, over the 48.000 GiB pool before any prefill; at the model's full window a cache is 5.060 GiB and ten of them is 50.6 GiB on its own. Two is a working default that leaves headroom for prefill; raise it to 4 if you actually run parallel sessions, or set --prompt-cache-bytes and let it trim.
• --host 127.0.0.1 is already the default on this server, unlike some MLX servers; it is written out so a copy-paste into a different tool does not quietly expose the endpoint.

A small cache and one sequence at a time also avoid a known kernel panic. ml-explore/mlx #3186 is open, carries an Apple Feedback number (FB22091885), and was reproduced on mlx_lm.server with mlx-lm 0.31.3 serving mlx-community/Qwen3.6-35B-A3B-4bit — the same architecture and the same affine 4-bit class as the builds on this page, not nvfp4 — at only ~20 GB resident on a 32 GB machine, so it is not a capacity problem you can outrun with 64 GB. One reporter isolated the trigger, and it is not prefill size: a faithful sequential replay of their own crashing session ran "111 minutes without a panic", while adding two concurrent request streams plus prompt-cache eviction churn panicked in "102–108 s, reproducibly (3/3 cold boots)". Their reading — "the trigger is concurrency + buffer-lifecycle churn" rather than prompt size or throughput — is a second, independent reason for the --prompt-cache-size 2 above and for driving this endpoint one conversation at a time. Two caveats if you read the thread. The mitigation that held up in that reporter's own A/B — never calling mx.set_wired_limit, ten clean runs against three stock panics — is a change to mlx-lm's behaviour, not a setting. And the ready-made shim a different operator posted a month later, which patches that call out of mlx.core, keys on an environment variable that is, in its author's own words, "not an mlx-lm feature, it is just this shim's own toggle" — nothing in mlx-lm reads MLX_LM_DISABLE_WIRED_LIMIT.

There is no repetition-penalty flag on either mlx_lm.server or mlx_lm.generate in this release. On the server it is a request-body parameter (repetition_penalty, default 0.0 = off), so the card's 1.05 goes in the JSON, not on the command line. Note also that the two entry points spell their template options differently: the server takes --chat-template-args, mlx_lm.generate takes --chat-template-config.

2. Drive it as an agent

The card is explicit that tool schemas go through the API's tools= parameter rather than being pasted into the prompt, "so the system prompt itself only needs to set the role and the high-level objective" — the Qwen3.5 chat template renders them into the <tool_call><function=...> form the model was trained on.

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "apodex-1.1-mini",
    "messages": [
      {"role": "system", "content": "You are Apodex, an AI assistant developed by Apodex AI. Current time: 2026-08-30."},
      {"role": "user", "content": "What is (25 + 15) * 3 - 10?"}
    ],
    "temperature": 1.0, "top_p": 0.95, "repetition_penalty": 1.05, "max_tokens": 2048
  }'

A one-shot run without a server, useful for a first smoke test and for the KV-quantisation flags the server does not expose:

mlx_lm.generate \
  --model ./apodex-1.1-mini-mlx-6bit \
  --max-tokens 2048 --temp 1.0 --top-p 0.95 \
  --prompt "Plan, step by step, how you would verify a claim in a PDF against its cited source."

mlx_lm.generate additionally accepts --kv-bits 8 --kv-group-size 64 --quantized-kv-start 5000, which roughly halves the KV cache — 5.000 GiB down to about 2.656 GiB at the full window, since MLX's affine 8-bit at group_size 64 costs 8.5 bits per element rather than 8. Note where that lever is and is not: mlx_lm.generate serves one prompt and holds one cache, and the server — which is where the multi-cache pool actually lives — has no --kv-bits at all, as noted above. So this shrinks a single one-shot run, and it cannot be used to buy back concurrency on the server path.

Alternative: llama.cpp-Metal, and why it is not the lead

Mainline llama.cpp registers this architecture — { LLM_ARCH_QWEN35MOE, "qwen35moe" }, sits at line 42 of src/llama-arch.cpp at release b10690 (published 2026-08-30, the tree every llama.cpp reference on this page was read at), Metal is on by default on macOS, and bartowski's GGUF repo publishes a wider quant ladder than MLX does. It is a real path, and this is a comparison between two imperfect ones rather than a dismissal. It is not the lead here because of one specific report:

Issue #27442 — open, labelled bug-unconfirmed, last updated 2026-08-25 — reports that qwen35moe hybrid models return an empty completion through llama-server on Apple Metal once the prompt passes roughly 16.5K tokens. The reporter, a community user, writes that "The native context length of these models is 262K, so being effectively hard-capped at ~16-19K via the server is a severe practical limitation." After a llama.cpp contributor suggested sampler noise, they re-tested at temperature 0 with a fixed seed on a build containing the subsequent Metal flash-attention rewrite and reported that "the EOS-first behaviour is deterministic, position-sensitive, checkpoint-specific". The contributor who replied was explicit about the limits of their own input: "We have not reproduced the bug — we run this architecture on CPU and ROCm, not Metal".

Two boundaries on that report, because they cut in opposite directions and a reader deserves both:

• It is not about this checkpoint. The two models that reproduce it are Ornith-1.5-35B-A3B and a Qwen3.6-35B-A3B IQ2_XXS. Nobody has published a test on an Apodex GGUF, either way.
• It does look Metal-shaped rather than architecture-shaped. A separate report of the same symptom on a 64-layer hybrid, #27756, reproduces on CUDA and CPU at a ~130K threshold and records that the same needle test at 243k tokens "PASSES on Qwen3.5-35B-A3B (30 GDN layers) on the same build, same server, same machine" — and 30 GDN layers with 10 full-attention layers is exactly this model's shape. So the family's long-context failures are not uniform, and the one that lands at 16K lands on Metal.

MLX is not the clean option, and this page will not pretend otherwise. Searching the mlx-lm tracker for qwen3_5 returns a page of issues on this architecture, and the open ones include a Metal OOM during long-context prefill on this exact architecture shape (#1480, quoted at the top of this page), unbounded server memory growth past the prompt-cache cap under multi-day serving (#1807), one-token completions on MTP variants when a system prompt is reused (#1292), and the two speculative-decoding dead ends this page's troubleshooting covers. Both runtimes have open hybrid-MoE defects on Apple.

The reason to lead MLX is that the defects are different in kind, and one kind is much worse to build an agent on. The llama.cpp Metal report is a silent wrong answer: an empty completion with predicted_n = 1 and a clean HTTP 200, at ~16.5K tokens, on a model whose whole point is long-horizon work — a harness cannot tell that from a model choosing to stop. The MLX reports are loud resource failures: an OOM that crashes, or a process footprint you can watch grow. Both cost you the session; only one of them lies about it. Add that the MLX threshold reported is an order of magnitude further out (160K succeeding, 176K failing, on a bigger machine) than the ~16.5K one, and the choice is clear enough to write down. If you want the GGUF path anyway, for the finer quant ladder or for work that stays short, verify your own long-prompt behaviour before trusting it:

brew install llama.cpp

hf download bartowski/apodex_Apodex-1.1-mini-GGUF \
  --include "apodex_Apodex-1.1-mini-Q6_K.gguf" \
  --local-dir ./apodex-gguf

llama-server -m ./apodex-gguf/apodex_Apodex-1.1-mini-Q6_K.gguf \
  -ngl 99 -c 65536 -np 1 --no-mmproj \
  --host 127.0.0.1 --port 8080 \
  --jinja --temp 1.0 --top-p 0.95

Three notes on that command. Download the single .gguf explicitly rather than passing a bare -hf …-GGUF: the repo also ships an mmproj projector, and a bare -hf picks it up and puts it in memory for a recipe that has no use for it — --no-mmproj refuses it a second time. Pass -c explicitly: fit_params still defaults to true on master (bool fit_params = true; in common/common.h at b10690), so an unset context is silently sized to fit the device rather than to your workload. And on long-running servers watch --cache-ram: #27894 was filed as a fixed per-request RSS growth on qwen35moe models and closed once the reporter established it was the prompt cache filling to its bound — the same class of surprise as --prompt-cache-size on the MLX path, one runtime over.

Not a path today: Ollama

There is no entry for this model in Ollama's official library. Checked 2026-08-30: ollama.com/library/apodex and ollama.com/library/apodex-1.1-mini both return 404, and registry.ollama.ai/v2/library/apodex/manifests/{latest,35b,apodex-1.1-mini} return 404 for all three tag names, while the same calls against qwen3 return 200. That covers the library/ namespace only — ollama run hf.co/bartowski/apodex_Apodex-1.1-mini-GGUF would still work, but on a Mac of this size it runs through the same Metal engine and inherits the caveat above.

Results

• Speed: omitted, because no measurement of this model on any Apple chip exists on the surfaces searched. That space, so you can judge the gap: all 21 repos the Hub returns for Apodex-1.1-mini on 2026-08-30, each card fetched as raw README and scanned for throughput units and for any named Apple machine. Twenty cards returned content and one (aykutx21) has no card at all — its README 404s and its whole tree is a single .gitattributes, so it is counted as unreadable rather than as a clean negative — and no card anywhere names an Apple machine — no M1–M5, no MacBook, Mac Studio or Mac mini — against a control confirming the same pattern does match that text when it is present. A targeted web search for this model on an M2 Max returned Apple-Silicon benchmark aggregators and the model's own card, and nothing for the pair. Treat that list as a snapshot: it is a young model and repos are still appearing, so re-run the enumeration rather than trusting the number. Measurements of this model do exist — just not on this silicon, and not in a form that transfers. kingjones777 publishes a four-build ladder of 64.87 / 63.30 / 45.05 / 32.54 tok/s, noting "All measured by me on a Ryzen AI MAX+ 395 (Strix Halo, gfx1151, ROCm 7.2.4) with the whole model on GPU", under a third-party ROCm fork of llama.cpp; ghazni101 reports 226 tok/s on an RX 7900 XTX under hipfire, a separate Rust engine with its own container format. Different vendor, different silicon, different runtimes, different quant families: neither bounds this page in either direction, and neither is quoted as if it did. What the first one does illustrate is that the tier axis this page reasons about is a real one on real hardware — his ordering is monotone in build size. If you run this, please contribute the measurement — mlx_lm.benchmark ships with mlx-lm and produces exactly the numbers /check/apodex-1-1-mini/m2-max is missing.
• Unified memory usage: two numbers, and the difference between them is the point. Resident at this page's 131,072-token window is 28.794 GiB against a 48.000 GiB addressable pool (26.919 GiB at 32,768; 31.294 GiB at the full 262,144) — derived from published artifact bytes and the model's config.json, not observed. The peak during a long prefill is larger and is the number that decides your ceiling: about 40.4 GB against a 51.54 GB pool at 131,072, rising to 52.0 GB [50.1–53.3] at the full 262,144 — over the line on the central estimate — extrapolated from the community ladder in mlx-lm #1480 and its raw per-rung data. Nobody has measured either figure on this pair.
• Quality notes: Apodex publishes agentic-evaluation scores for this model (50.2 on FrontierFinance, 27.7 on APEX-Agent). Those are the vendor's own figures from the vendor's own harness, and they say nothing about this machine or about any quantisation of it — treat them as a vendor claim, not a measurement. The effect of 6-bit versus 4-bit on them is unmeasured by anyone, which is stated in the quant argument above rather than papered over: this page spends memory on fidelity because the memory is free here, not because someone measured the gain.

For the full benchmark data, see /check/apodex-1-1-mini/m2-max.

Troubleshooting

Answers are short, flat, or stop mid-thought

Three server defaults, all corrected on the command line above. --max-tokens defaults to 512 and this model spends most of a turn inside <think>, so a truncated reasoning block is the common cause of a "wrong" answer. --temp defaults to 0.0 and --top-p to 1.0, against the card's recommended 1.0 / 0.95 — greedy decoding on a reasoning model tends to flatten and loop. And repetition_penalty has no CLI flag at all: pass it in the request body (the card recommends 1.05; the server's default is 0.0, i.e. off).

The model insists it is Apodex, or ignores your persona

That is the chat template, not the weights. chat_template.jinja sets identity_mode to 'fallback' when the caller does not define it, and injects a # Role Definition block naming Apodex whenever identity_mode == 'force' or identity_mode == 'fallback' with no caller system message. Send your own system message and the injection stops; pass identity_mode: 'force' through the template args if you want it regardless.

Memory climbs across a few different conversations

The prompt cache, almost certainly. At 6-bit this machine holds eight caches at this page's 131,072-token window — and fewer once a prefill is in flight — against a default of ten. The server keeps up to --prompt-cache-size distinct KV caches and applies no byte cap unless --prompt-cache-bytes is set, so several long conversations accumulate at up to 2.560 GiB each at 131,072, or 5.060 GiB each at the full window. The server logs Prompt Cache: N sequences, X GB at INFO — read that before blaming the weights. Batched decode is the other multiplier: --decode-concurrency defaults to 32, and every figure in this recipe is for a single sequence.

If it keeps climbing after you have capped the cache, that is a known open report and not your configuration. mlx-lm #1807, filed 2026-08-30 against a hybrid-attention qwen3_5 model on mlx-lm 0.31.3, describes a server on a 128 GB Mac Studio whose footprint passes 30 GB for a ~16 GB model under an 8 GB cache cap, over multi-day serving, reclaimed only by restarting the process. No maintainer has replied yet and no workaround is offered, so on a long-lived agent endpoint the practical mitigation today is to watch the footprint and restart on a schedule. A closed sibling report, #1641, traced a related growth to ArraysCache.advance() leaking Metal buffer objects in qwen3_5 batch decode — the recurrent cache this architecture uses on 30 of its 40 layers is the common thread in both.

Speculative decoding requires a trimmable prompt cache

Speculative decoding does not work for this architecture on mlx-lm, and the reason is structural rather than a misconfiguration. mlx_lm/generate.py refuses when any layer's cache reports itself non-trimmable — "Speculative decoding requires a trimmable prompt cache" — and this model's 30 linear layers use ArraysCache, which defines neither is_trimmable nor trim — at v0.31.3 and on main alike — and therefore inherits _BaseCache's return False, while KVCache and its siblings in the same file define both. A recurrent state cannot be rolled back the way a KV cache can. The open issue is mlx-lm #1446, whose reporter hit it on exactly this shape — a Qwen 35B target with a small draft model through mlx_lm.server; the pull request that would have fixed it by making the recurrent cache trimmable, #1730, was closed unmerged on 2026-08-14.

Multi-token prediction is not an alternative route to it on this path: the first-party checkpoint ships an MTP head, mlx-lm drops it on load, and all five MLX builds of this model carry zero mtp. tensors, so there is no head left to draft with. Even a checkpoint that kept one would not help through this server — mlx-lm #1462 is open on mlx_lm.server --draft-model failing to load any Qwen3.6 MTP checkpoint at all, because model_type qwen3_5_mtp is not registered. There is an Apple-native runtime built specifically to use such heads — MTPLX, Apache-2.0, which lists Qwen 3.5/3.6/3.8 and Gemma 4 — but its published catalogue contains no Apodex build, and the family entry it does ship at this size is a Qwen 3.6 35B MoE rather than this model. So the accurate statement is narrower than "there is nothing": there is no ready speculative-decoding path for this model on Apple today, and the thing that would have to change first is an MTP-retaining Apple build of it.

Garbage output from an MLX build whose name ends in -mtp

Avoid those builds — on the release this page installs. At v0.31.3, mlx-lm's sanitize decides whether a checkpoint still needs its RMSNorm shift with should_shift_norm_weights = has_mtp_weights or has_unsanitized_conv1d, using the presence of MTP tensors as a proxy for "this is a raw export". For an already-converted checkpoint that keeps its MTP head the proxy is wrong, every norm is shifted a second time, and the model generates token salad with no error raised anywhere.

This is fixed upstream but not in a release yet, and the dates are the whole story. mlx-lm #1623 — "Fix Qwen3.6 converted RMSNorm double shift" — was merged on 2026-08-18, and it drops the MTP proxy in favour of the raw Conv1D layout alone. Reading mlx_lm/models/qwen3_5.py on main confirms it: should_shift_norm_weights and has_mtp_weights no longer occur in the file, while the "mtp." not in k filter remains. Two other attempts at the same fix, #1442 (still open) and #1735 (closed in favour of #1623), were not merged. But v0.31.3 shipped on 2026-04-22, four months before the merge, so the release you install above still has the hazard and will until the next one. The four builds this page tabulates all carry zero mtp. tensors and are unaffected either way.

A build's card warns of a Metal OOM below 48 GB

Every nicolasembleton tier carries the same one-sentence Known Issues section: "Metal OOM during long-context prefill on less than 48 GB unified memory. Keep context under 32k tokens." Read the qualifier — less than 48 GB — and then check what the sentence is attached to. Diffing the four cards against each other, the only differences are the tier name, the bits-per-weight and the size line, so this caution is byte-identical on an 18.169 GiB build and on a 34.299 GiB one. A warning that does not change across a 16 GiB spread in artifact size is not a measurement of any of them, and it names no chip, no context length it was observed at, and no figure.

It is still worth taking seriously in one respect, and that is the word prefill — because there is a real report underneath the boilerplate. mlx-lm #1480 documents precisely this failure on this architecture shape, and it does so on a 128 GB Mac Studio, which is the detail that matters: the card's "less than 48 GB" threshold is not where the wall is. The machine that hit it had twice this machine's addressable pool.

And the thread does not settle where the wall is, which is worth knowing before you plan around it: the issue is a 128 GB machine failing at 176k, while the ladder comment on it is a different 128 GB machine completing 176k and 192k. Same capacity, opposite outcomes — so the wall is not a fixed function of how much memory the machine has. What sets it is unresolved, and the thread's own "very likely headroom" reading does not survive the arithmetic: at the failing length this model's derived peak is about 36.5 GB against roughly 103 GB addressable on a 128 GB machine, so a residency explanation needs some 67 GB held by something else, and the reporter states the persistent cache was small at the time. The two runs also differ in mlx version and code path, either of which is a likelier candidate than raw headroom. So the card is directionally right about the failure mode and wrong about the quantity, which is what you would expect from a sentence that is byte-identical across four builds.

What neither the card nor the issue supports is a fit verdict on this machine — the resident budget above is arithmetic and it clears. If you see a prefill OOM, shorten the prompt first — the transient tracks prompt length, and lowering --prefill-step-size was measured on this architecture not to bound it (the peak-memory section has the numbers). And please report it — that would be the first Apple datapoint on this model either way.

Tried to install FlashAttention, a CUDA toolkit, or the NVFP4 build

None of those apply. There is no CUDA on macOS, no FP8 or NVFP4 tensor hardware on Apple Silicon, and no GPTQ/AWQ/bitsandbytes GPU kernels — MLX uses its own Metal kernels and its own affine quantisation. The first-party Apodex-1.1-mini-NVFP4 and -GPTQ-Int4 repos are for NVIDIA deployments. Note what does not disqualify them, because it is the thing people repeat: their quick-starts are single-GPU — both launch SGLang at --tp-size 1, and the sibling FP8 repo at --tp 1 — so they are not eight-GPU datacenter recipes you can wave away on scale. It is the unquantised base repo whose quick-start runs eight-way, at --tp 8 and --tensor-parallel-size 8, on the 71.90 GB bf16 weights. The quantised three fail here for the only reason that matters on this machine: NVFP4, GPTQ and FP8 are CUDA-side formats with no Apple Silicon implementation, at any tensor-parallel degree. If a generic tutorial tells you to pass --tensor-parallel-size 8, -DGGML_CUDA=ON, or to pip install flash-attn, skip it — the commands above are the complete Apple path.

Report anything else via the submission form.