Apodex 1.1 mini on Apple M4 Max: a 262K-context agent server on MLX

What You'll Build

An OpenAI-compatible agent endpoint running Apodex 1.1 mini — Apodex AI's reasoning-first agent model, a 35B Mixture-of-Experts post-trained from Qwen/Qwen3.5-35B-A3B — entirely on a MacBook Pro M4 Max, served by mlx_lm.server at the model's full 262,144-token context, with enough room left over that you never touch a wired-limit override. The interesting part is the runtime choice: mainline llama.cpp registers this architecture and will load the GGUFs, but a Metal-specific defect against this exact architecture makes it the wrong lead on a Mac today. That argument is spelled out below rather than asserted.

Hardware data: Apple M4 Max (48 GB unified memory, 546 GB/s) · working set 23.229 GiB at the full 262,144-token context, against a 36.000 GiB addressable pool · See benchmark data

ℹ️ Unified memory is not VRAM. Apple Silicon has no dedicated VRAM. Our catalogue stores full unified memory in the GPU row, but Metal's recommendedMaxWorkingSetSize caps what the GPU may actually address: exactly 3/4 on machines of 36 GB and above, so a 48 GB Mac addresses 36.000 GiB and not 48. Every figure below is framed against 36.000 GiB, in GiB throughout, and never against the marketing 48.

ℹ️ This is a text-only recipe, and that is a property of the runtime, not a hedge. mlx-lm's loader for this architecture discards the checkpoint's vision weights before the model is built — qwen3_5_moe.py skips any key that key.startswith("vision_tower") or key.startswith("model.visual") — and every published MLX build of this model is a tensor census of 1,757 tensors with zero vision keys and zero mtp. keys, against 1,811 / 333 / 785 in the first-party checkpoint. So the memory budget below carries no perception encoder and no multi-token-prediction head, because the artifact you download does not contain them. Apodex's own card is pipeline_tag: text-generation and documents no image input.

ℹ️ An MoE keeps every expert resident — the file size is the memory cost. This model activates 8 of 256 experts per token, which is a compute property. All 256 stay in memory, so the footprint is the whole quantised file (19.509 GB at 4-bit), not some smaller "active" fraction.

Requirements

Which M4 Max this is. Apple sells the M4 Max in two bins and the memory size identifies the bin: per Apple's MacBook Pro specifications, 36 GB unified memory goes with the 14-core-CPU part at 410 GB/s, while 48, 64 and 128 GB are sold only on the 16-core-CPU / 40-core-GPU part at 546 GB/s. A 48 GB M4 Max is therefore unambiguously the 546 GB/s bin. This matters because token generation on Apple Silicon is memory-bandwidth-bound.

The memory budget (derived, not measured)

Nothing here was observed on a machine. Every input is a byte count from the Hugging Face tree API or a field read out of the model's own config.json, and the arithmetic is shown so you can redo it.

The architecture is hybrid: of its 40 decoder layers only some carry a context-scaling KV cache. config.json declares full_attention_interval: 4 and a layer_types array that counts out to 30 linear_attention and 10 full_attention entries; mlx-lm derives the same split independently from self.is_linear = (layer_idx + 1) % args.full_attention_interval != 0, and allocates accordingly — [ArraysCache(size=2) if l.is_linear else KVCache() for l in self.layers]. So only 10 of 40 layers hold a cache that grows with context. That is the whole reason a 262K window is discussable on a laptop.

Per-token KV is 2 (K + V) × 2 KV heads × 256 head dim × 2 bytes = 2,048 B — 2 KiB per full-attention layer, which is 20 KiB/token across the 10 of them: 32,768 tokens costs 0.625 GiB and the full 262,144 costs 5.000 GiB. The 2-byte element is the model's bfloat16: mlx_lm.server has no KV-quantisation flag (--kv-bits exists only on mlx_lm.generate), so on the server path the cache is at model precision.

The recurrent half does not grow with context. Each linear layer holds a Gated-DeltaNet state of 32 value heads × 128 × 128 at float32 (state = mx.zeros((B, Hv, Dv, Dk), dtype=mx.float32)) = 2.000 MiB, plus a (kernel−1) × conv_dim = 3 × 8192 bfloat16 conv state = 48 KiB. Thirty layers of that is 0.060 GiB, constant, whether you run 1K or 262K.

Consequence: no sudo sysctl iogpu.wired_limit_mb raise is needed on this machine, and you should not add one. The full-context working set sits 12.771 GiB inside the default pool.

What this budget does not count. It is the resident set — weights, KV cache, recurrent state. It does not include transient prefill activations, which on a 256-expert MoE scale with --prefill-step-size (default 2048 on the server) rather than with total context. On a 48 GB Mac the 12.771 GiB of headroom absorbs that comfortably; nearer the 36 GB floor it is the term most likely to be the difference between fitting and not, and lowering --prefill-step-size is the lever.

Why 36 GB is the floor. The same arithmetic, run against every Apple memory configuration:

36 GB is a real shipping configuration, not a rounded-off number — Apple's specification page lists it as a sold SKU. That is where min_vram_gb: 36 comes from: it is a filter floor naming a machine you can buy on which the install documented here runs at the context this recipe documents, not a measured peak.

Installation

1. Install mlx-lm

The architecture is implemented in stock mlx-lm — mlx_lm/models/qwen3_5_moe.py is present at tag v0.31.3, which is the current PyPI release. No --trust-remote-code, no git checkout, no third-party model file:

python3 -m venv .venv && source .venv/bin/activate
pip install -U "mlx-lm>=0.31.3"

There is nothing CUDA-shaped to install and nothing to configure for the GPU: MLX targets Metal directly. Skip pip install flash-attn, skip any cu12x wheel index, and ignore the first-party NVFP4 / GPTQ-Int4 / FP8 builds on the Hub — those are CUDA-native formats (NVFP4 is a Blackwell microscaling format, GPTQ kernels are CUDA-only, FP8 tensor cores are Hopper/Blackwell) and Apple Silicon has none of that hardware.

2. Download the 4-bit MLX weights

pip install -U huggingface_hub

hf download abenzerps/Apodex-1.1-mini-MLX --local-dir ./apodex-1.1-mini-mlx

mlx_lm.server --model abenzerps/Apodex-1.1-mini-MLX also resolves the repo id directly and downloads into the Hugging Face cache; the explicit hf download above is only for controlling where ~20 GB lands.

Which build, and the quant ladder. Two independent authors published a 4-bit conversion of this model on the same day and they came out byte-identical in size — 19,509,024,201 B across 4 shards, 1,757 tensors — so this is a reproducible mlx_lm.convert output rather than one person's idiosyncratic build. Both declare group_size: 64, bits: 4, mode: affine with 80 per-tensor 8-bit overrides, which are the 40 layers' mlp.gate and mlp.shared_expert_gate routers. A finer ladder exists from one of them:

On a 48 GB M4 Max the 6-bit build is the fidelity upgrade this machine can actually take: 31.294 GiB at the full context still clears the pool by 4.706 GiB. The 8-bit build does not fit at any useful context without raising the wired limit, and is listed here so you do not reach for it by reflex. Every one of these is a single-author community conversion. Enumerating all 76 repos the Hub returns for a search on Apodex, these five are the only MLX builds of the 1.1 mini — there is no mlx-community build of it, so the usual "prefer the org build" shortcut has nothing to point at here and the provenance question is decided by the artifacts themselves.

The MLX build's chat_template.jinja is byte-identical to the first-party one (both 8,831 B), so it carries no template drift — worth checking on any redistributor, and here it passes.

Running

1. Start the server

mlx_lm.server \
  --model ./apodex-1.1-mini-mlx \
  --host 127.0.0.1 --port 8080 \
  --temp 1.0 --top-p 0.95 \
  --max-tokens 32768 \
  --prompt-cache-size 1

Every one of those flags is doing real work, and three of them are correcting a default that will otherwise bite you:

• --temp 1.0 --top-p 0.95. Apodex's card gives temperature: 1.0, top_p: 0.95, repetition_penalty: 1.05, max_tokens: 32768 as its recommended agentic sampling. mlx_lm.server's own default is "Default sampling temperature (default: 0.0)" — greedy — with --top-p at 1.0. You have to pass them.
• --max-tokens 32768. The server default is 512. This is a reasoning model that opens every turn with a <think> block; a 512-token ceiling truncates mid-thought and returns something that looks like a wrong answer rather than a truncation.
• --prompt-cache-size 1. The default is 10, described in the parser as "Maximum number of distinct KV caches to hold in the prompt cache", and --prompt-cache-bytes — the only byte cap — is unset by default, so nothing trims by size unless you ask. Ten full-context caches at 5.000 GiB each is 50 GiB against a 36.000 GiB pool. Either hold one, or set --prompt-cache-bytes explicitly.
• --host 127.0.0.1 is already the default here, unlike some MLX servers; it is written out so a copy-paste into a different tool does not quietly expose the endpoint.

There is no repetition-penalty flag on either mlx_lm.server or mlx_lm.generate in this release. On the server it is a request-body parameter instead (repetition_penalty, default 0.0 = off), so the vendor's 1.05 goes in the JSON, not on the command line.

2. Drive it as an agent

The card is explicit that tool schemas go through the API's tools= parameter rather than being pasted into the prompt, "so the system prompt itself only needs to set the role and the high-level objective" — the Qwen3.5 chat template renders them into the XML-ish tool_call / function / parameter element form the model was trained on, which the card documents in full.

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "apodex-1.1-mini",
    "messages": [
      {"role": "system", "content": "You are Apodex, an AI assistant developed by Apodex AI. Current time: 2026-08-28."},
      {"role": "user", "content": "What is (25 + 15) * 3 - 10?"}
    ],
    "temperature": 1.0, "top_p": 0.95, "repetition_penalty": 1.05, "max_tokens": 2048
  }'

A one-shot run without a server, useful for a first smoke test and for the KV-quantisation flags the server does not expose:

mlx_lm.generate \
  --model ./apodex-1.1-mini-mlx \
  --max-tokens 2048 --temp 1.0 --top-p 0.95 \
  --prompt "Plan, step by step, how you would verify a claim in a PDF against its cited source."

mlx_lm.generate additionally accepts --kv-bits 8 --kv-group-size 64 --quantized-kv-start 5000, which roughly halves the 5.000 GiB full-context cache. You do not need it on a 48 GB machine; it is the lever for the 32 GB tier.

Alternative: llama.cpp-Metal, and why it is not the lead

Mainline llama.cpp does register this architecture: { LLM_ARCH_QWEN35MOE, "qwen35moe" } is at line 42 of src/llama-arch.cpp, and llm_arch_is_hybrid includes it, so the recurrent/attention split is modelled rather than ignored. Metal needs no flag — the build docs say "On MacOS, Metal is enabled by default. Using Metal makes the computation run on the GPU." — and bartowski's GGUF repo publishes a far wider quant ladder than MLX does (IQ4_XS 19.28 GB / 17.955 GiB, Q4_K_M 21.86 GB / 20.363 GiB, Q6_K 30.53 GB / 28.430 GiB, Q8_0 37.81 GB / 35.217 GiB), with ?expand[]=gguf confirming the baked architecture really is qwen35moe at context_length 262144.

The problem is specific, current, and on this platform. Issue #27442, open since 2026-08-20 and last updated 2026-08-25, reports that qwen35moe hybrid models return an empty completion through llama-server on Apple Metal once the prompt passes roughly 16K tokens. The reporter — a community user, not a maintainer — writes that "The native context length of these models is 262K, so being effectively hard-capped at ~16-19K via the server is a severe practical limitation." After a llama.cpp contributor suggested the run-to-run variation might be sampler noise, they re-tested at temperature 0 with a fixed seed on a build containing the subsequent Metal flash-attention rewrite and reported that "the EOS-first behaviour is deterministic, position-sensitive, checkpoint-specific" — and that a different Gated-DeltaNet hybrid generates normally at every position where this architecture fails, up to 48K. Two qwen35moe checkpoints reproduce it; nobody has published a test on an Apodex GGUF either way.

That is a report against the architecture, not against this model, and it is unconfirmed by a maintainer. But a 262K-context agent model whose whole selling point is long-horizon work is exactly the workload the report says breaks, and MLX has a first-party implementation with no equivalent open report, so MLX leads here. If you want the GGUF path anyway — for the finer quant ladder, or for work that stays short — it is a reasonable second choice, and you should verify your own long-prompt behaviour before trusting it:

brew install llama.cpp

hf download bartowski/apodex_Apodex-1.1-mini-GGUF \
  --include "apodex_Apodex-1.1-mini-Q4_K_M.gguf" \
  --local-dir ./apodex-gguf

llama-server -m ./apodex-gguf/apodex_Apodex-1.1-mini-Q4_K_M.gguf \
  -ngl 99 -c 65536 --host 127.0.0.1 --port 8080 \
  --jinja --temp 1.0 --top-p 0.95

Download the single .gguf explicitly rather than passing a bare -hf …-GGUF: that repo also ships mmproj-apodex_Apodex-1.1-mini-f16.gguf (899,283,488 B), and a bare -hf will pick a projector up and put it in memory for a recipe that has no use for it.

Not a path today: Ollama

There is no entry for this model in Ollama's official library. Checked 2026-08-28: ollama.com/library/apodex returns 404, and registry.ollama.ai/v2/library/apodex/manifests/{latest,35b,apodex-1.1-mini} return 404 for all three tag names, while the same two calls against qwen3 return 200. That is the library/ namespace only — ollama run hf.co/bartowski/apodex_Apodex-1.1-mini-GGUF would still work, but it runs the same GGUF through the same llama.cpp Metal engine and inherits the caveat above.

Results

• Speed: omitted, because no measurement of this model on any Apple chip surfaced. The space searched, so you can judge the gap: every repo the Hub returned for Apodex-1.1-mini when this was enumerated on 2026-08-28 — 21 of them, the four first-party ones, both MLX publishers' five builds and twelve GGUF or other third-party conversions — checked on both surfaces a measurement is published on, each fetched individually rather than sampled. Their discussions tabs hold exactly one thread between them (a vendor announcement). Their model cards were fetched as raw README and grepped for throughput units and for any named Apple machine: 20 cards returned content, one (aykutx21) has no README at all, and no card anywhere names an Apple machine — no M1–M5, no MacBook, Mac Studio or Mac mini. That list is a snapshot and it moves: two of the 21 were published within 24 hours of this writing, so re-run the enumeration rather than trusting the number. Measurements of this model do exist — just not on this silicon. Two publishers report their own throughput figures on AMD hardware: kingjones777 on a Ryzen box under ROCm with a GGUF, and ghazni101 on an RX 7900 XTX under hipfire, a Rust inference engine using its own .mq4r container. Neither figure is quoted here: different vendor, different silicon, and in the second case a runtime that shares nothing with MLX, so neither bounds this page in either direction. The remaining surfaces: the full issue list of ApodexAI/FrontierAgent (21 items, enumerated, none hardware-related); the llama.cpp and mlx-lm trackers, where a search for "Apodex" returns zero against controls that return 156 and 378; and two web searches. A Reddit search could not be completed — the request was refused for the control query as well as the real one, so that space is untested rather than empty. If you run this, please contribute the measurement — mlx_lm.benchmark ships with mlx-lm and produces exactly the numbers /check/apodex-1-1-mini/m4-max is missing.
• Unified memory usage: 23.229 GiB at the full 262,144-token context against a 36.000 GiB addressable pool, derived from published artifact bytes and the model's config.json rather than observed. At 32,768 tokens the same arithmetic gives 18.854 GiB.
• Quality notes: Apodex publishes agentic-evaluation scores for this model (50.2 on FrontierFinance, 27.7 on APEX-Agent). Those are the vendor's own figures on the vendor's own harness, three of the six benchmarks in the family being Apodex creations, and they say nothing about this machine — treat them as a vendor claim, not a measurement. The 4-bit quantisation's effect on them is unmeasured by anyone; the 6-bit build above is the cheap way to buy that uncertainty down on a 48 GB Mac.

For the full benchmark data, see /check/apodex-1-1-mini/m4-max.

Troubleshooting

Answers are short, flat, or stop mid-thought

Three server defaults, all correctable on the command line above. --max-tokens defaults to 512 and this model spends most of a turn inside <think>, so a truncated reasoning block is the common cause of a "wrong" answer. --temp defaults to 0.0 and --top-p to 1.0, against the card's recommended 1.0 / 0.95 — greedy decoding on a reasoning model tends to flatten and loop. And repetition_penalty has no CLI flag at all: pass it in the request body (the card recommends 1.05; the server's default is 0.0, i.e. off).

The model insists it is Apodex, or ignores your persona

That is the chat template, not the weights. chat_template.jinja sets identity_mode to 'fallback' when the caller does not define it, and injects a # Role Definition block naming Apodex whenever identity_mode == 'fallback' and not has_caller_system. Send your own system message and the injection stops; pass identity_mode: 'force' through the template args if you want it regardless.

Speculative decoding requires a trimmable prompt cache

Speculative decoding does not work for this architecture on mlx-lm, and the reason is structural rather than a misconfiguration. speculative_generate_step raises "Speculative decoding requires a trimmable prompt cache" when any layer's cache reports itself non-trimmable, and this model's 30 linear layers use ArraysCache, which implements neither is_trimmable nor trim — at v0.31.3 and on main alike. A recurrent state cannot be rolled back the way a KV cache can. The open issue is mlx-lm #1446; the pull request that would have fixed it, #1730, was closed without being merged.

Multi-token prediction is not an alternative route to it here either: the first-party checkpoint ships an MTP head (785 tensors), and mlx-lm drops it on load — weights = {k: v for k, v in weights.items() if "mtp." not in k} — so no MLX build of this model carries one. The sibling package mlx-vlm does ship drafter machinery for this architecture family, but it consumes a separately published MTP drafter repo, and no such artifact exists for this model in the 76-repo Hub enumeration — the only -mtp-suffixed Apodex 1.1 build is a full checkpoint that retains its head, which is the build the next entry tells you to avoid. Do not go looking for a --draft-model speed-up on this path; there is not one to find.

Garbage output from an MLX build whose name ends in -mtp

Avoid those builds. mlx-lm's sanitize decides whether a checkpoint still needs its RMSNorm (1 + w) shift with should_shift_norm_weights = has_mtp_weights or has_unsanitized_conv1d — using the presence of MTP tensors as a proxy for "this is a raw export". For an already-converted checkpoint that keeps its MTP head the proxy is wrong, every norm is shifted a second time, and the model generates token salad with no error raised anywhere. That is the failure described in mlx-lm #1442 and #1735, neither of which is merged. Scope: this is the behaviour of v0.31.3, the release you install above. On mlx-lm's main the proxy is already gone — should_shift_norm_weights and has_mtp_weights no longer appear in qwen3_5.py at all, while the "mtp." not in k filter remains — so expect this hazard to disappear at the next release while the reader-facing advice stays the same either way. At least one MTP-retaining community build of this model exists on the Hub. The four builds recommended above all carry zero mtp. tensors and are unaffected.

A build's card warns of a Metal OOM below 48 GB

Open any of the four nicolasembleton tiers and its entire Known Issues section is a single sentence: "Metal OOM during long-context prefill on less than 48 GB unified memory. Keep context under 32k tokens." Read it in full — the qualifier in the middle is the load-bearing part — then check what it is attached to. The four cards are one template — diffing the 4-bit against the 8-bit shows the only differences are the tier name, the bits-per-weight and the size line, so this caution is byte-identical on an 18.169 GiB build and on a 34.299 GiB one. A warning that does not change across a 16 GiB spread in artifact size is not a measurement of any of them, and it names no chip, no context length it was observed at, and no figure.

It is worth taking seriously in one respect, though, and that is the word prefill. The budget above counts resident memory and not the transient activations a prefill step allocates — see the note there — so a card author who hit a wall during long-context prefill on a smaller machine may well be reporting something real that no weights-plus-KV arithmetic predicts. What the sentence does not support is a fit verdict: on the 48 GB machine this recipe documents, its own threshold is not crossed, and the 36 GB floor here is derived per-build from that build's bytes rather than inherited from a template. If you are near the floor and see a prefill OOM, lower --prefill-step-size before lowering -c, and please report it — that would be the first Apple datapoint on this model either way.

Out of memory after a few different conversations

The prompt cache, almost certainly. The server keeps up to --prompt-cache-size distinct KV caches (default 10) and applies no byte cap unless --prompt-cache-bytes is set, so several long conversations accumulate at up to 5.000 GiB each. The server logs Prompt Cache: N sequences, X GB at INFO — check it before blaming the weights. Batched decode is the other multiplier: --decode-concurrency defaults to 32, and the budget in this recipe is for a single sequence.

Tried to install FlashAttention, a CUDA toolkit, or the NVFP4 build

None of those apply. There is no CUDA on macOS, no FP8 or NVFP4 tensor hardware on Apple Silicon, and no GPTQ/AWQ/bitsandbytes GPU kernels — MLX uses its own Metal kernels and its own affine quantisation. The first-party Apodex-1.1-mini-NVFP4 (24.21 GB) and -GPTQ-Int4 (24.65 GB) repos are for NVIDIA deployments; the card's own quick-start is an eight-way tensor-parallel SGLang/vLLM launch, which is a datacenter recipe and not applicable here at all. If a generic tutorial tells you to pass --tensor-parallel-size 8, -DGGML_CUDA=ON, or to pip install flash-attn, skip it — the commands above are the complete Apple path.

Report anything else via the submission form.