Fara1.5-9B on RTX 5070: a Blackwell Browser Computer-Use Agent with llama.cpp

What You'll Build

A local computer-use agent on a single RTX 5070: llama-server hosting Fara1.5-9B as a Q5_K_M GGUF plus its vision projector, and Microsoft's fara-cli harness pointed at it, driving a Playwright-launched Chromium at 1440×900. You give it a task in English; it takes a screenshot, reasons, and emits a click, a keystroke or a URL visit, then loops.

Hardware data: RTX 5070 (12GB VRAM) · Q5_K_M + f16 mmproj + 32K context = 8.434 GiB accountable, 9.434 GiB once this recipe's flash-attention-off score buffer is counted · See benchmark data

⚠️ This model clicks, types and submits in a real browser, and the default path here is not a sandbox. That the model acts is the product, not a side effect. The model card's Responsible AI section asks you to "run Fara in an isolated container with no access to host files, environment variables, or sensitive credentials", and the same card states plainly: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." fara-cli launches its own Chromium through Playwright with a fresh browser context — it does not attach to your logged-in browser — but it runs uncontained, on your machine, with your network. Isolation is a separate step you have to take: see Sandbox it below, and do it before pointing this at anything that matters.

ℹ️ Not a general-purpose vision chatbot. Fara1.5-9B is fine-tuned for one loop: screenshot in, <tool_call> out. It is in our multimodal vertical because it takes images and text and emits text, but it is trained against a specific system prompt and a fixed computer_use action schema. Used as a plain image-captioning model it will still answer, but nothing about it is tuned for that. Per the model card: "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree."

⚠️ Runtime status — read before you commit an evening to this. llama.cpp support for this architecture is verifiable statically, and this recipe shows that evidence, but no published issue thread, user report or docs entry shows Fara1.5-9B captioning an image under llama-server or llama-mtmd-cli. The qwen35 text arch and the qwen3vl_merger projector are both first-class in llama.cpp master, and four independent quantisers have converted this model's vision tower successfully — but that is static evidence, not a tested path, and on the newest consumer architecture that gap is wider than usual: nothing in the llama.cpp tracker reports this model on Blackwell either way. Treat the first run as the test.

What the tracker does hold, filed on Blackwell silicon, is llama.cpp #25717: an open report of llama-server dying with CUDA error: an illegal memory access was encountered on vision requests with an mmproj loaded. It concerns a different model — a 35B mixture-of-experts sibling of this dense one — and nobody has reproduced it on any Fara build. But the failure it describes lands in machinery this model shares, so the serve command below passes --flash-attn off and the vision check runs two tests rather than one. That flag is not free: it costs a gigabyte of VRAM at this recipe's context, which the budget below accounts for line by line. Troubleshooting sets out precisely what transfers and what does not.

Requirements

Component	Minimum	This recipe
GPU	12GB VRAM, CUDA compute 12.0 (sm_120); 9.434 GiB accountable with flash attention off	— not measured; budget below is derived (/contribute)
RAM	16GB (model is mmap'd; browser + Playwright on top)	—
Storage	7.771 GB decimal / 7.237 GiB (Q5_K_M 6.382 GiB + f16 mmproj 0.855 GiB)	~8 GiB with Chromium
Software	CUDA Toolkit 12.8+, llama.cpp b10087+, Python 3.10+, Playwright	—

Why the 9B, and not the 4B or the 27B

The 9B is the middle of three sizes and has to earn the tier. Microsoft answers this directly in the Fara1.5 paper: "Fara1.5-9B already covers a substantial fraction of the 4B-to-27B gain on both metrics, which makes it the natural choice when deployment cost is a constraint." — followed immediately by "Fara1.5-27B is the right choice when raw accuracy matters more than cost." On a 12 GB card, cost is the constraint, so this is the tier.

The numbers behind that, from the paper's Table 7 (averaged over three independent runs, with 95% confidence intervals — a better source than the bare figures on the model card):

Model	WebVoyager	Online-Mind2Web	WebTailBench v1.5
Fara1.5-4B	80.8 ± 2.3	57.3 ± 4.3	27.4 ± 2.8
Fara1.5-9B	86.6 ± 2.0	63.4 ± 4.3	32.3 ± 2.8
Fara1.5-27B	89.3 ± 1.8	72.3 ± 3.8	40.2 ± 3.0

Microsoft's own reading of those intervals is that they are "small relative to the gaps between model sizes, confirming that the scaling trend in Table 3 is not an artifact of run-to-run noise" — and across the family as a whole that is fair. Doing the arithmetic pairwise gives a more granular picture, and this next part is our reading, not theirs: on WebVoyager the 9B reaches 88.6 at the top of its interval while the 27B drops to 87.5 at the bottom of its own, so the two overlap; on Online-Mind2Web (67.7 vs 68.5) and WebTailBench (35.1 vs 37.2) they do not. Overlapping 95% intervals is only a conservative proxy for a significance test and does not by itself establish that two results are indistinguishable — so read this as the WebVoyager gap is much the weakest-separated of the three, not as a null result. The 27B's advantage is real but not uniform, and it is smallest exactly where the benchmark is easiest.

The paper is specific about where that gap comes from, and it is not speed. Larger models "solve more of the hard tail", and on success rate versus trajectory length: "The mapping is largely independent of size in the bulk of the step range; larger models retain a higher success rate only in the long tail." Nor is the 9B less efficient per task it completes — "The larger mean step counts of smaller models above are largely a consequence of their higher failure rate, not lower efficiency." Expect the 9B to handle ordinary tasks about as directly as the 27B would, and to lose ground specifically on long, hard trajectories. And the ceiling is low for everyone on the hardest benchmark — the paper's own words are that "even the 27B model leaves a substantial fraction of examples unsolved in three attempts" there.

This exact question was asked on the 9B's own HuggingFace discussions in July 2026 — "Does anyone have any suggestions on the kinds of tasks where the 9B model fails where the 27B model appears to be necessary?" — and has no community answer beyond the paper. Nobody has published a qualitative rule; the interval arithmetic above is as sharp as the public evidence gets.

What the tier costs on this card is unusually easy to state, because the 4B and 9B are architecturally identical everywhere except width. The 9B's config.json declares num_key_value_heads: 4, head_dim: 256, num_hidden_layers: 32, num_attention_heads: 16 and full_attention_interval: 4, exactly as the 4B does — so the per-token KV cost of the 9B is exactly the same as the 4B's, so is its fixed recurrent state, and so is the attention-score buffer derived below. The entire difference is weights: 3.088 GiB → 6.382 GiB at Q5_K_M, plus 0.626 → 0.855 GiB of projector, from the 4B and 9B GGUF trees. You pay ~3.5 GiB of otherwise-idle VRAM for those points and nothing at all in running cost.

The 27B is simply out of reach here: its Q4_K_M is 16.329 GiB before the 0.864 GiB projector.

How it fits in 12 GB — the arithmetic

There is no benchmark for this pair yet (/check reports unknown), so every figure below is derived from the artifact and from llama.cpp's own source. Reproduce it rather than trusting it. All totals are GiB (2^30 bytes), which is the unit the card is really in — a "12GB" NVIDIA GPU has 12 GiB of physical memory — so they compare directly. HuggingFace displays file sizes in decimal GB, which is why the Storage row above and the weights below describe the same bytes in different units.

Weights. From the bartowski GGUF tree: Fara1.5-9B-Q5_K_M.gguf = 6,852,928,640 B = 6.382 GiB (HuggingFace displays this as 6.85 GB — decimal), mmproj-Fara1.5-9B-f16.gguf = 918,166,048 B = 0.855 GiB.

KV cache — smaller than you expect, because this is a hybrid model. config.json carries a layer_types array of 24 linear_attention + 8 full_attention entries, and llama.cpp agrees: llm_arch_is_hybrid(LLM_ARCH_QWEN35) returns true in src/llama-arch.cpp, and src/models/qwen35.cpp assigns layers with is_recr_impl[i] = (i + 1) % full_attn_interval != 0. With full_attention_interval = 4 and 32 blocks that is layers 3, 7, … 31 — 8 attention layers, exactly the indices config.json lists. Only those get a growing cache:

K+V per token per attention layer = 2 × n_head_kv(4) × head_dim(256) × 2 B (f16) = 4,096 B
× 8 attention layers                                                            = 32 KiB / token

n_ctx = 16,384 → 0.500 GiB
n_ctx = 32,768 → 1.000 GiB
n_ctx = 65,536 → 2.000 GiB

A non-hybrid 32-layer model with the same head geometry would cost four times that.

Recurrent state — fixed in context, but four times bigger than one sequence's worth. The other 24 layers hold a context-independent state. Using llama.cpp's n_embd_r() / n_embd_s() in src/llama-hparams.cpp with this file's own GGUF keys (ssm.conv_kernel 4, ssm.inner_size 4096, ssm.group_count 16, ssm.state_size 128 — which config.json sources as linear_conv_kernel_dim, linear_num_value_heads × linear_value_head_dim, linear_num_key_heads and linear_key_head_dim):

conv state : (4−1) × (4096 + 2×16×128) =  24,576 elems
ssm  state :        128 × 4096         = 524,288 elems
per layer  : 548,864 × 4 B (F32)       = 2.094 MiB
× 24 recurrent layers                  = 50.25 MiB   ← per sequence
× n_seq_max = 4                        = 201.0 MiB = 0.196 GiB

That last multiplication is the one everybody drops. src/llama-model.cpp sizes the recurrent memory with /* recurrent_rs_size */ std::max((uint32_t) 1, cparams.n_seq_max), and tools/server/server.cpp resolves the server's auto slot count to params.n_parallel = 4; params.kv_unified = true; whenever n_parallel < 0 — which is the default it gets from common/arg.cpp (params.n_parallel = -1; // auto by default for the server example). So llama-server allocates four sequences' recurrent state whether or not you ever use more than one, and the honest figure is 0.196 GiB, not 0.049. It is a small term either way; it is here because the arithmetic on this page is meant to be reproducible against a real startup log — and because the same n_parallel = 4 default is load-bearing again two sections down.

Subtotal at the recommended settings:

	GiB
Q5_K_M weights	6.382
f16 mmproj	0.855
KV cache @ 32,768	1.000
Recurrent state (4 sequences)	0.196
Accountable subtotal	8.434

The flash-attention-off surcharge, and why it changes the sums on this card

Everything above is architecture and applies to every card that runs this model. This next term is specific to the configuration this recipe recommends on Blackwell, and it is the largest single number on the page that is not weights.

With flash attention enabled, the attention score matrix never exists as a tensor. With it disabled, src/llama-graph.cpp takes its other branch — ggml_tensor * kq = ggml_mul_mat(ctx0, k, q); — and then forces the result to full precision with ggml_mul_mat_set_prec(kq, GGML_PREC_F32), under the source's own comment that "this op tends to require high floating point range". That is an explicit n_kv × n_ubatch × n_head tensor in f32, reserved in the compute buffer. With this model's 16 attention heads and llama.cpp's default -ub 512:

32,768 × 512 × 16 × 4 B = 1,073,741,824 B = 1.000 GiB

It scales linearly in both -c and -ub, which is what makes it dangerous to inherit rather than derive:

Context	kq bound at -ub 512	at -ub 256
16,384	0.500 GiB	0.250 GiB
32,768	1.000 GiB	0.500 GiB
65,536	2.000 GiB	1.000 GiB
131,072	4.000 GiB	2.000 GiB

Total for this recipe: 8.434 + 1.000 = 9.434 GiB, leaving 2.566 GiB of a 12 GiB card for the CUDA context, the remaining compute buffers, the vision encode graph and whatever the browser holds. That is workable but it is not the 3.566 GiB of slack the same configuration enjoys with flash attention on, and it is the reason this page leads with a 32,768 context rather than the 65,536 the KV arithmetic alone would happily allow: doubling the window here costs 1.000 GiB of KV and another 1.000 GiB of kq, which is 11.434 GiB and no longer a sensible margin.

Why min_vram_gb is 12 and not something smaller. The floor is the smallest real card capacity on which the install documented here actually runs, and the answer has to be computed after the surcharge, not before it. 9.434 GiB does not fit an 8 GB card at any context worth running — even 16,384 with -ub 256 leaves 8.184 GiB against 8 GiB of physical memory — so 12 GB is the smallest capacity that works, and 12 is the value in the frontmatter. It is worth showing the working because this exact term has already broken a floor elsewhere in this model family: on a card whose lead context is 131,072 the surcharge is 4.000 GiB rather than 1.000, which pushed a 16 GiB configuration over its card and forced the floor up a tier. If you change -c, re-derive it.

What is still not in the tables. The CUDA context, the rest of the compute buffers and the vision graph are allocated at load time and this recipe does not derive them. Neither is the browser: Playwright launches a real Chromium on this machine, and if it renders on the GPU its process holds memory on the same 12 GB. Read the CUDA0 compute buffer size lines that llama-server prints on startup, watch nvidia-smi with the agent actually running, and report what you see.

How the context is actually spent

A CUA sends a screenshot every step, so the per-image cost decides whether the budget above is realistic — and then the round count decides the rest of it.

The harness fixes the viewport at 1440×900 (Fara15AgentConfig.viewport_width/height in src/fara/agents/fara/fara15_agent.py) and pre-resizes with Qwen's smart_resize at factor = patch_size × merge_size = 32 → 1440×896. llama.cpp then applies the identical algorithm (calc_size_preserved_ratio in tools/mtmd/mtmd-image.cpp) and clamps to 8–4096 merged tokens for this projector (set_limit_image_tokens(8, 4096) in tools/mtmd/clip.cpp); 1,290,240 px sits inside that, so nothing is rescaled:

(1440 / 32) × (896 / 32) = 45 × 28 = 1,260 tokens per screenshot
× max_n_images = 3                 = 3,780 tokens resident

Microsoft's own harness budgets image_token_estimate: 1500 per image, so 1,260 is the real figure and 1,500 is their conservative rounding.

So the image half is capped — the card is explicit that only the most recent three screenshots are kept, and max_n_images: int = 3 in the harness config enforces it. What is not capped is the text trajectory: every round appends the model's reasoning, its tool call and an observation (max_observation_chars: int = 1000), and none of that is evicted. So the round count, not the screenshot count, is what decides your -c:

32,768 − 3,780 resident screenshots ≈ 29,000 tokens of trajectory room
  / --max_rounds  15  (the smoke test below)  ≈ 1,900 tokens per round
  / --max_rounds 100  (the CLI default)       ≈   290 tokens per round

290 tokens per round is thin for a model that writes its reasoning out before every action — and the default is the thin case, not the roomy one: src/fara/run_fara.py declares --max_rounds with default=100. llama.cpp will not paper over the overrun either. common/common.h ships bool ctx_shift = false; // context shift on infinite text generation, so a trajectory that outgrows the window errors rather than quietly sliding it — an unattended run stops mid-task rather than degrading.

And here this page parts company with its flash-attention-on siblings: cap --max_rounds, do not raise -c. On a card that leaves flash attention alone, the answer to a thin per-round budget is -c 65536 for 1.000 GiB of extra KV. With the surcharge this recipe carries, that same move costs the KV and another 1.000 GiB of kq — the 11.434 GiB the section above already ruled out. So keep -c 32768 and set the round count deliberately: divide the ~29,000 tokens of trajectory room by the --max_rounds you intend to pass and satisfy yourself the quotient covers one round of reasoning plus a tool call. If it does not, the task is too long for this configuration — shorten the task, or give it to the agent in stages. Widening the window is the expensive lever here, and -ub 256 (see Troubleshooting) buys back only half the surcharge, not the KV.

The harness README's 15,000 tokens is a floor rather than a target, and a floor is not a recommendation for this card; the arithmetic above is.

Installation

1. Build llama.cpp with CUDA 12.8+ for sm_120

This is the one step that genuinely differs on Blackwell. The RTX 5070 is compute capability 12.0 per NVIDIA's CUDA GPUs table, and llama.cpp's CUDA CMakeLists annotates that target as "Blackwell, needs CUDA v12.8, FP4 tensor cores". Check the toolkit first:

nvcc --version    # must report release 12.8 or newer

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

Two details worth knowing rather than pasting:

• Pass 120 explicitly rather than trusting the default. Left alone, llama.cpp does not assemble a fixed architecture list at all: the same CMakeLists sets CMAKE_CUDA_ARCHITECTURES to native whenever GGML_NATIVE is on, and ggml/CMakeLists.txt defaults GGML_NATIVE ON. So the default targets whichever card happens to be attached at compile time. Naming the architecture yourself is what makes the build reproducible on a machine that is not the one the GPU sits in — and it keeps an inadequate toolkit failing at configure time, where you can see it.
• You write 120, the build system compiles 120a. The same file rewrites any plain 12X architecture into its architecture-specific 12Xa form, because — in its own words — "the Blackwell FP4 tensor core instructions are not forwards compatible". You do not need to type the a, and you should not be surprised to see 120a-real in the CMake status line. Note that this rewrite is unconditional, so its appearance proves nothing about your toolkit version; use nvcc --version for that.

Build from a llama.cpp release at or after b10087 — that is the release the GGUF used here was quantised on. Support for this model is first-class in mainline at both halves, and it is worth confirming before you download 7 GiB:

grep -n 'LLM_ARCH_QWEN35,' src/llama-arch.cpp                      # -> { LLM_ARCH_QWEN35, "qwen35" }
grep -n 'qwen3vl_merger' tools/mtmd/clip-impl.h                    # -> PROJECTOR_TYPE_QWEN3VL
grep -n 'Qwen3_5ForConditionalGeneration' conversion/qwen3vl.py    # -> Qwen3VLVisionModel (the mmproj converter)

A note on NVFP4, since Blackwell is why people ask. The runtime side is not the gap: GGML_TYPE_NVFP4 is a first-class quant type in ggml/include/ggml.h, and ggml/src/ggml-cuda/mmq.cu selects a native FP4 matmul for it behind blackwell_mma_available(cc), a gate this card passes. The gap is that there is nothing to feed it: of the eleven Fara1.5-9B repositories on the Hub, none is an NVFP4 build. So build for 120 because that is your architecture; the quantised path on this page is a GGUF K-quant through llama.cpp's ordinary CUDA kernels.

2. Download the weights and the projector

There is no first-party GGUF — Microsoft ships safetensors only. Use a community quantization; four independent quantizers publish a matching mmproj for the 9B (bartowski, prithivMLmods, DevQuasar, Tdamre), which is itself corroboration that the vision conversion works.

pip install -U huggingface_hub
hf download bartowski/Fara1.5-9B-GGUF Fara1.5-9B-Q5_K_M.gguf --local-dir ./fara-gguf
hf download bartowski/Fara1.5-9B-GGUF mmproj-Fara1.5-9B-f16.gguf --local-dir ./fara-gguf

Downloading the projector is not optional. In GGUF the vision tower lives in a separate file loaded by tools/mtmd; without it you have a text-only model and a computer-use agent that cannot see. You can verify the artifact you just downloaded declares the projector llama.cpp implements:

strings ./fara-gguf/mmproj-Fara1.5-9B-f16.gguf | grep -m1 qwen3vl_merger

3. Install the fara harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Omit the [vllm] extra from the README's command — that path is for bf16 serving on datacenter cards and is not what this recipe uses.

4. Point the harness at your local server

cat > local_llamacpp.json <<'EOF'
{
    "model": "Fara1.5-9B",
    "base_url": "http://127.0.0.1:5000/v1",
    "api_key": "sk-no-key-required"
}
EOF

Running

Start the server

./build/bin/llama-server \
  -m ./fara-gguf/Fara1.5-9B-Q5_K_M.gguf \
  --mmproj ./fara-gguf/mmproj-Fara1.5-9B-f16.gguf \
  --flash-attn off \
  --reasoning-format none \
  --image-min-tokens 1024 \
  -c 32768 \
  -ngl 99 \
  --temp 0 \
  --host 127.0.0.1 --port 5000

Four of those flags are load-bearing and none of them is a default:

• --flash-attn off — the Blackwell-specific one, spelled out rather than left alone because the default is the setting under suspicion. common/arg.cpp declares the flag as [on|off|auto] and fills its help text from the struct default, which is auto; src/llama-context.cpp then computes cparams.flash_attn = params.flash_attn_type != LLAMA_FLASH_ATTN_TYPE_DISABLED, so auto starts enabled and is only downgraded if a support probe says otherwise. Flash attention on is the one variable in #25717's reproduction matrix that failed unconditionally — including on a nineteen-token prompt, and including with the vision encoder moved to the CPU. Turning it off removes that condition. It does not make you immune, and Troubleshooting is explicit about why; it also costs the 1.000 GiB derived above.
• --reasoning-format none — mandatory. See Troubleshooting; without it the agent loop silently receives nothing. common/arg.cpp documents this value as "none: leaves thoughts unparsed in message.content", which is exactly what the harness parses.
• --image-min-tokens 1024 — llama.cpp's default floor for this projector is 8 merged tokens, and tools/mtmd/clip.cpp warns that "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks". A full 1440×900 screenshot is 1,260 tokens and clears the floor on its own, but any smaller image the agent sends would not.
• -c 32768 — the harness README requires "Please ensure that context length is set to at least 15000 tokens and temperature to 0 for best results." With flash attention off, raising it costs twice what the KV table alone suggests. See the surcharge section.

And two flags deliberately not added. --kv-unified is tempting, because both surviving configurations in #25717 ran a unified cache — but on llama-server it is already the default and adding it changes nothing (arg.cpp sets params.n_parallel = -1 for the server example; server.cpp turns that into n_parallel = 4 and kv_unified = true), and the issue's row 6 passes it explicitly and still crashes. The instruction that actually matters is the inverse: do not add --parallel. It is the one flag present in both flash-attention-off rows that crashed — row 5, where an explicit value leaves kv_unified at its false default and the cache is split, and row 6, where --kv-unified forces the cache back to unified and the crash happens anyway. Read that pair the honest way round: a unified cache is not a rescue either, so the case against --parallel is that it is present in every long-prompt failure in the matrix and buys you nothing here. It also divides your window, since src/llama-context.cpp sets cparams.n_ctx_seq = cparams.n_ctx only when the cache is unified and cparams.n_ctx / cparams.n_seq_max otherwise.

If you would rather not manage files, -hf bartowski/Fara1.5-9B-GGUF:Q5_K_M fetches the model and, per llama.cpp's server docs, the matching mmproj alongside it.

Prove the vision path works before going further

First read three lines of the startup log: the CUDA0 device line (it must name your 5070 — if the model loaded on CPU, go back to step 1), the flash-attention line (it must report disabled; if it does not, your flag did not take), and the mmproj load stage.

Then check that the server advertises multimodal and send it a real image. llama.cpp's server README tells clients to check /models or /v1/models for the multimodal capability before sending images, and documents image_url as a first-class content part.

Test A — is the projector wired up at all?

curl -s http://localhost:5000/v1/models | grep -o multimodal

curl -s http://localhost:5000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "Fara1.5-9B",
  "temperature": 0,
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "Describe this image in one sentence."},
    {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}
  ]}]
}' | python3 -m json.tool

If the reply describes the picture, the projector is wired up. If it answers as though no image were present, or the server never logged an mmproj load, stop here — the agent loop below will silently produce nonsense actions rather than fail loudly. On a freshly-released architecture this step earns its keep twice over: it separates "my build has no kernels for this card" from "the model is bad at this".

Test B — and on Blackwell this is the one that earns its keep. Test A puts only a handful of text tokens in front of the image, which is the shape #25717 reports as passing once flash attention is off. The shape it reports as still crashing is a long text prefix ahead of the image, and a real agent prompt is the long one. Probe it before you trust the setup:

python3 - <<'PY'
import json, urllib.request
long_text = "Describe precisely. " * 200          # ~800 tokens of text, BEFORE the image
body = {"model": "Fara1.5-9B", "temperature": 0, "max_tokens": 128,
        "messages": [{"role": "user", "content": [
            {"type": "text", "text": long_text},
            {"type": "image_url", "image_url": {"url":
                "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}]}]}
req = urllib.request.Request("http://localhost:5000/v1/chat/completions",
                            data=json.dumps(body).encode(),
                            headers={"Content-Type": "application/json"})
print(urllib.request.urlopen(req, timeout=600).read().decode()[:400])
PY

--image-min-tokens 1024 floors even that small logo at 1,024 image tokens, so the image batch is a realistic size rather than a toy. If Test B returns a description you have cleared both reported triggers, on a build newer than any in the report. If Test B kills the server while Test A passed, you have reproduced #25717 on a new model and a new architecture enum — look for find_slot: non-consecutive token position above the CUDA error in the log, read the Troubleshooting entry, and please post it upstream.

Run a task

source .venv/bin/activate
fara-cli \
  --task "find the current UTC time on time.is and tell me" \
  --endpoint_config local_llamacpp.json \
  --output_folder ./run1 \
  --save_screenshots \
  --headful \
  --max_rounds 15

--max_rounds 15 is a smoke-test value, not the default. run_fara.py declares that argument with default=100, so dropping the flag gives you a hundred rounds against a window this card cannot widen cheaply — see How the context is actually spent. Cap it deliberately: on this configuration the round count is the lever, not -c.

--headful reads like a convenience flag and is not one here. The harness runs the browser headless by default: src/fara/run_fara.py declares the flag as "Run the browser in headful mode (show GUI, default is headless)" and passes headless=not args.headful into the browser launch. Omit it and a real Chromium still opens, still clicks and still submits — you simply cannot watch it. With the flag, the command above opens a real, unsandboxed Chromium on your desktop, which is what makes the supervision advice on this page true rather than aspirational.

Each step writes a screenshot and a data_point.json trajectory into --output_folder. The model stops on its own at critical points — entering personal data, payments, submissions, sign-ins — and asks rather than proceeding.

Sandbox it

fara-cli gives you a fresh Chromium context, not container isolation. For anything beyond a first smoke test, run the harness inside a container with no host mounts and no credentials in its environment, or use Magentic-UI, which the model card names as the recommended deployment and which the harness README describes as "a sandboxed browser environment with auditable action logging and user prompts at critical points". Match the sandbox viewport to the trained resolution — the card names 1440×900 as the resolution Fara is most commonly trained with, and is explicit about what to do with that: "Match this in your sandbox for the most reliable grounding."

Results

• Speed: omitted. No benchmark of Fara1.5-9B on an RTX 5070 — or any consumer GPU — exists at the time of writing, and Microsoft publishes no throughput figures at all: the card names "A6000, A100, H100, and B200" only as hardware its bf16 serving path was tested on, which says nothing about a Q5_K_M build on Blackwell. Blackwell is also the architecture with the least community data for this model family: no NVFP4 build of this model exists to exercise the card's native FP4 path, and the one published measurement of this architecture's SSM kernels was taken on much older silicon (see Troubleshooting). Note too that this recipe deliberately runs with flash attention disabled, which costs prompt-processing throughput — so if you do contribute a number, say whether you measured it with the flag as documented or with flash attention on. Please contribute your numbers so this line can be filled in from real data rather than extrapolation.
• VRAM usage: 8.434 GiB of accountable allocations at Q5_K_M / 32K context, plus a 1.000 GiB attention-score buffer that exists only because flash attention is off — 9.434 GiB for this recipe as documented. All of it derived above from file sizes and llama.cpp's own memory formulas; compute buffers beyond the kq term, the CUDA context and the browser's own GPU memory sit on top and are not derivable from static inspection. Track the empirical picture at /check/fara1-5-9b/rtx-5070.
• Quality notes: the vision path here is structurally proven, not runtime-proven. Four independent facts establish that it loads: mainline llama.cpp registers Qwen3_5ForConditionalGeneration for both the text model and the vision tower (conversion/qwen3vl.py); the shipped mmproj declares clip.projector_type = qwen3vl_merger, which maps to PROJECTOR_TYPE_QWEN3VL in tools/mtmd/clip-impl.h; four quantizers converted it independently; and llama-server documents image_url content parts, which is exactly what the harness sends. What does not exist is a published report of Fara1.5 captioning an image or grounding a click under llama-server, on this architecture or any other. The vendor's published scores are for the bf16 model on datacenter hardware; Q5_K_M is a quantised approximation of those weights, and no published evaluation measures how much grounding accuracy a given tier costs.
• Language: English only, per the model card's out-of-scope list.

For the full benchmark data, see /check/fara1-5-9b/rtx-5070.

Troubleshooting

CUDA error: an illegal memory access was encountered on a vision request

This is the Blackwell failure worth knowing about, and it is why the serve command disables flash attention and the vision check has two tests. llama.cpp #25717 reports llama-server dying with exactly that message on vision requests, preceded in the log by find_slot: non-consecutive token position 774 after 773 for sequence 0 with 2040 new tokens — 2,040 being the image token batch. The reporter bisected seven configurations on an RTX 5070 Ti 16 GB (Blackwell, sm_120), driver 610.74 — one tier up from this card, same silicon. Two of the seven completed: flash attention off with the default unified cache and a short (~19-token) text prefix, and one where the same long prompt was sent with the image as the first content part. Every row with flash attention on crashed, including the one with the nineteen-token prompt and including one that ran the vision encoder on the CPU via --no-mmproj-offload. The reporter's reading of the mechanism is that "Qwen3.5's M-RoPE 3D position ids are mishandled by the FA path; and the split-KV path requires consecutive positions which M-RoPE image batches violate."

What transfers, precisely. The report is about a different model, and the honest summary is same machinery, different model, unverified outcome. On the transfer side:

• Fara1.5-9B genuinely uses M-RoPE. Its config.json carries rope_parameters.mrope_section = [11, 11, 10] with mrope_interleaved: true, and the GGUF's qwen35.rope.dimension_sections reads [11, 11, 10, 0] — the same section layout the reported model's GGUF carries.
• Both load a qwen3vl_merger projector at patch_size = 16, spatial_merge_size = 2. Same projector implementation; only the tower's depth and width differ.
• The warning that precedes the crash is emitted by src/llama-memory-recurrent.cpp — the recurrent half of the hybrid cache, which 24 of this model's 32 layers pass through. That check exists because the architecture is hybrid. It is neither MoE-specific nor size-specific, which is the strongest argument for taking the report seriously here.

Against the transfer:

• The reported model registers as qwen35moe — a different llama.cpp architecture enum from this model's qwen35 — with 40 layers to 32 and 2 KV heads to 4, and it is a mixture-of-experts model where this one is dense.
• Every row of the matrix ran 24 layers' experts on the CPU (--n-cpu-moe 24) with -b 4096 -ub 4096. A fully-offloaded dense model at llama.cpp's defaults (-b 2048 -ub 512) is a different placement and a different batch split — and the reporter himself lists #19929, "batch-size dependent M-RoPE weirdness", as related.
• It is one report from one person with no maintainer reply. The thread's only comment is his own follow-up; no pull request references it, so nothing has been fixed — and nothing has been ruled out either. Its newest build, b10016, is described in the report as "current latest" on 15 July, older than the b10087 floor this recipe builds from, and nobody has re-tested since.

What this recipe does about it. --flash-attn off removes the one condition that crashed unconditionally; it is the removal of a necessary condition, not protection, because rows 5 and 6 crashed with it off. Test B probes the long-prefix trigger before you commit a run. One piece of luck is on your side — the harness puts the screenshot first in every user turn and the text after it (content=[ImageObj…, text_prompt] in fara15_agent.py, at both call sites), which is the ordering of the row that completed, and which is worth more to you than any flag. But Fara's system prompt is long text and still precedes that image in the flattened sequence, and the text history grows every round while the image history stays capped at three, so the prefix in front of each screenshot only gets longer. Test B is your detector; the first agent run is the experiment.

There is one earlier Blackwell precedent, worth reading for the variables it names rather than as corroboration: llama.cpp #21564, "Xid 43 GPU crash on RTX 5090 (Blackwell) after b8680 — flash_attn_stream_k_fixup kernel", which a bisect pinned to a flash-attention kernel change and which closed with nobody else having reproduced it on a newer driver and toolkit. Its durable lesson is that driver and toolkit version were the axis that moved the outcome — so record both if you end up filing anything.

Out of memory at load, or shortly after the first screenshot

Do not reach for the usual --cache-type-k q8_0 --cache-type-v q8_0 here. With the --flash-attn off this recipe needs, a quantised V cache is not a slow path or a warning — it is a refusal to start. src/llama-context.cpp rejects the pair outright with if (ggml_is_quantized(params.type_v) && params.flash_attn_type == LLAMA_FLASH_ATTN_TYPE_DISABLED), logging "quantized V cache requires flash_attn to be enabled" and returning a null context. It is the most-copied OOM remedy on the internet and it is the one thing you must not paste here.

Quantising the K half alone is legal without flash attention: block_q8_0 is one ggml_half scale plus 32 int8_t quants — 34 bytes per 32 values against f16's 64 — and head_dim of 256 divides that block size cleanly. Per token, across the 8 full-attention layers:

K: 8 × 4 × 256 = 8,192 elems -> f16 16.0 KiB  ->  q8_0 (8,192/32) × 34 B = 8.5 KiB
V: 8 × 4 × 256 = 8,192 elems -> f16 16.0 KiB  ->  unchanged             = 16.0 KiB
total per token                     32.0 KiB  ->                          24.5 KiB

In descending order of what the levers actually buy you on this card:

Lever	Frees	Costs you
-c 16384	1.000 GiB (0.500 KV + 0.500 kq)	half the window; still above the harness's 15,000-token floor
--no-mmproj-offload	0.855 GiB	slower image encoding, once per agent turn, on the CPU
-ub 256	0.500 GiB of kq	prompt-processing throughput — it halves the score-matrix reservation
--cache-type-k q8_0	0.234 GiB	a little K-cache precision; legal with flash attention off
Q4_K_M instead of Q5_K_M	0.877 GiB	grounding fidelity, on a model whose job is pixel coordinates

-ub 256 is the one to reach for first if the card is also driving your display: it is cheaper than dropping a quant tier on a model that predicts exact coordinates, and the kq reservation is the single largest term this recipe adds. Note that --no-mmproj-offload frees real VRAM but does not dodge #25717 — the issue's row 3 ran the vision encoder on the CPU and crashed anyway. Keep the f16 projector rather than a q8_0 one: at 0.855 GiB it is not where the savings are, and it trades vision fidelity in a model whose entire job is looking at pixels.

The agent does nothing — every step logs an empty response or list index out of range

This is the single failure everyone hits, and it is not a bug in your setup. Fara opens a <think> block and never closes it. Under llama-server's default reasoning extraction the parser therefore treats the whole output — reasoning and the <tool_call> block together — as reasoning, returns it in reasoning_content, and leaves message.content empty. The harness parses message.content for <tool_call>, finds nothing, and the loop dies.

Launch with --reasoning-format none, which llama.cpp's own help text describes as leaving thoughts unparsed in message.content. The raw output then begins with a stray unclosed <think> tag, which is cosmetic.

Reported for Fara1.5-27B by a community user on llama.cpp b10107 in the model's HF discussions and acknowledged there by Microsoft — swhitehead (a Fara org member) replied "Would you mind posting this as an issue on the Fara1.5 github repo so we can mention this in the hosting instructions?", which became microsoft/fara#82, still open. It applies to the 9B for a checkable reason rather than by assumption: the 9B's own GGUF carries the same chat template, whose generation prompt emits <think>\n with no closing tag whenever enable_thinking is not explicitly false. The same reporter adds: "If you have a router/proxy in front doing its own reasoning extraction, disable it there too."

The build succeeds but the GPU is barely used, or the server falls back to CPU

Suspect the toolkit before anything else, and check it directly: nvcc --version must report 12.8 or newer. Blackwell's FP4 tensor-core instructions are not forwards-compatible, so an older toolkit cannot emit sm_120a code at all. Do not try to confirm the toolkit from the configure log — the 12X → 12Xa rewrite in the CUDA CMakeLists is unconditional, so 120a appears in the resolved architecture list on every toolkit version and proves nothing. If the GPU really is idle with a build that configured and compiled, look past CUDA to -ngl and to whether llama-server logged the CUDA device at startup.

Clicks land in the wrong place

Two things to check before blaming the model. First, the viewport must be 1440×900 — the harness's default, and the resolution the card says to match. Second, confirm --image-min-tokens 1024 is set; llama.cpp's default floor for this projector is 8 merged tokens, and grounding is exactly what that warning is about.

If both are right, there is one more thing to rule out before you blame the quantisation. A community user reported that llama-server degrades fine-grained vision accuracy for PROJECTOR_TYPE_QWEN3VL models relative to llama-cli on the same build, with identical prompt-token counts — llama.cpp issue #22785, filed against the Vulkan backend at release b8545. Read it with three caveats: a maintainer disputed the reporter's bisect and asked for a proper one, the issue was closed as stale and unconfirmed, and nobody re-tested it on a CUDA build or anywhere near b10087. Its value here is as a diagnostic, not a known defect: push the same screenshot through llama-cli — the binary the reporter compared against, and one of the three that accept --mmproj — and compare what the model says about it.

Generation is slower than a 9B model should be

There is an open, community-reported performance regression against exactly the SSM kernels this architecture leans on — llama.cpp issue #25162 reports a 24–42% loss on an RTX 2080 Ti (Turing, SM75) after commit 9e58d4d69, and notes that the Qwen35 architecture uses linear-attention layers for about 75% of its blocks, which matches the 24-of-32 split derived above. Three caveats before you act on it: it has no maintainer response; it was measured only on Turing, several architecture generations before this card's Blackwell (SM120), and nobody has posted numbers for any Blackwell part; and the one candidate fix raised in the thread, PR #25185, was merged on 2026-07-01 with a stated scope of flash-attention kernels rather than the SSM path, with no re-measurement posted since. Release b10087 already contains that merge — though with the --flash-attn off this recipe sets, a fix scoped to flash-attention kernels cannot be doing anything for you either way. Treat it as a lead to check with llama-bench across a couple of builds, not as a known property of this card.

rope.dimension_sections has wrong array length; expected 4, got 3

This is a real open llama.cpp bug (PR #25334), but it belongs to Ollama-exported qwen35 blobs, not to the GGUF this recipe uses. The bartowski artifact's own metadata reads qwen35.rope.dimension_sections = [11, 11, 10, 0] — four entries — so it cannot hit this failure. If you see this error, you are loading a different file than the one downloaded in Step 2. There is no Ollama library entry for Fara1.5 in any form, so there is no supported path that produces such a blob.

Running it against your own accounts

Don't. The model is trained to pause before sign-ins, payments and submissions, and that training is a safety net, not a guarantee — the card lists prompt injection from page content among its known risks, and a vision-only agent has no way to distinguish a legitimate button from a hostile one that renders like it. Keep it in a container, on an allow-list, with no credentials in reach.