Fara1.5-4B on RTX 5070: Blackwell Browser Computer-Use Agent at Q8_0 with llama.cpp

What You'll Build

A local browser computer-use agent (CUA): Microsoft's Fara1.5-4B served by llama-server on an RTX 5070, with the vision projector loaded so the model actually sees browser screenshots, driven by Microsoft's own fara-cli harness against a plain local Playwright Chromium.

Hardware data: RTX 5070 (12GB VRAM) · Q8_0 + f16 mmproj = 4.812 GiB of weights, 7.008 GiB total at 65,536 context (derived below) · See benchmark data

⚠️ This model clicks, types and submits in a real browser — and fara-cli does not sandbox it. That is not boilerplate; it is the operating requirement, and it is the one thing to get right before anything else on this page. The model card is blunt: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." Fara also reads whatever is on the page, so a hostile page is a prompt-injection surface.

Be clear about what the tooling gives you. fara-cli drives a plain local Playwright Chromium — no container, no allow-list, no pause control. The controls the card asks for belong to Magentic-UI / MagenticLite: the card attributes "the browser runs in a Docker container with no access to host files or environment variables" to that deployment and then says "If you integrate Fara1.5-4B directly, you're responsible for these controls." The harness README describes Magentic-UI as "a sandboxed browser environment with auditable action logging and user prompts at critical points" and offers it as what you use "instead of fara-cli you can use Magentic-UI". The vendor's position on the CLI path is a recommendation, not a guarantee shipped in it: "We recommend running it in a sandboxed environment, monitoring its execution, and avoiding sensitive data or high-risk domains."

So if you follow the fara-cli path below, you are the isolation layer. Practically: run it on a throwaway OS user or VM, let Playwright use its own fresh browser profile (never your daily one), stay logged out of everything, keep tasks on low-stakes public sites, and watch the run rather than leaving it unattended. If you want the container, allow-lists, watch-mode and pause the model card asks for, use Magentic-UI instead — see Running.

ℹ️ Vision is the product, not a bonus. Per the model card, "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree." A Fara install that only serves text is a broken install — the --mmproj step below is mandatory, and Step 4 exists to prove an image really reached the model before you let it drive anything.

⚠️ Runtime status — read before you commit an evening to this. llama.cpp support for this architecture is verifiable statically and this recipe shows that evidence, but no published issue thread, user report or docs entry shows Fara1.5 specifically captioning an image under llama-server or llama-mtmd-cli. The qwen35 text arch and the qwen3vl_merger projector are both first-class in llama.cpp master, and four independent quantisers have converted this model's vision tower successfully — but that is static evidence, not a tested path. On the newest consumer architecture that gap is wider than usual: nothing in the llama.cpp tracker reports this model on Blackwell either way. Treat the first run as the test. There is also no published speed measurement for this pair (see Results).

What the tracker does hold, filed on Blackwell silicon, is llama.cpp #25717: an open, unanswered report of llama-server dying with CUDA error: an illegal memory access was encountered on vision requests, with an mmproj loaded, on an RTX 5070 Ti (Blackwell, sm_120). It is a different model — a 35B mixture-of-experts sibling of this 4B dense one — and nobody has reproduced it on any Fara build. But the failure it describes lands in machinery this model shares, so Step 3 now passes --flash-attn off and Step 4 runs two vision tests rather than one. Troubleshooting sets out precisely what transfers and what does not; read it before you decide how much of your evening to risk.

Requirements

Component	Minimum	This recipe
GPU	12GB VRAM, CUDA compute 12.0 (sm_120)	not measured — contribute a run
RAM	16GB	—
Storage	5.17 GB (Q8_0 + mmproj)	5.17 GB decimal / 4.812 GiB
Software	CUDA Toolkit 12.8+, llama.cpp b10087+, Python 3.10+, Playwright	—

The BF16 safetensors total 8.455 GiB across two shards per the HF tree API. Microsoft's own Quickstart targets datacenter hardware — the card's requirements line says "A6000, A100, H100, and B200 have been tested", and its highlights claim only that the model "runs on a single A100/H100/B200 with room for the screenshot history". The consumer route is the quantised one documented here.

What 12 GB actually buys — the budget, derived

There is no benchmark for this pair yet, so every number below is arithmetic from the artifact's own metadata rather than a measurement. All totals are GiB (2^30 bytes), which is the unit the card is really in — a "12GB" NVIDIA GPU has 12 GiB of physical memory — so they compare directly. HuggingFace reports file sizes in decimal GB, which is why the Storage row above (5.17 GB) and the weights total below (4.812 GiB) describe the same bytes.

Weights — byte counts from the bartowski GGUF tree:

Component	Bytes	GiB
Fara1.5-4B-Q8_0.gguf	4,493,954,144	4.185
mmproj-Fara1.5-4B-f16.gguf	672,423,584	0.626
Weights total	5,166,377,728	4.812

KV cache — and this is where Fara1.5 is unusual. Its qwen35 architecture is a hybrid: llama.cpp's llm_arch_is_hybrid() in src/llama-arch.cpp returns true for LLM_ARCH_QWEN35, and llama-memory-hybrid.cpp splits layers with the filter !hparams.is_recr(il). The GGUF reports qwen35.block_count = 32 and qwen35.full_attention_interval = 4, and the source config.json spells the split out layer by layer: 24 of 32 layers are linear_attention, only 8 are full_attention (indices 3, 7, 11 … 31). So only 8 layers carry a KV cache that grows with context.

With qwen35.attention.head_count_kv = 4 and attention.key_length = attention.value_length = 256:

per layer per token = (4 x 256) + (4 x 256)      = 2,048 elements
x 8 full-attention layers                        = 16,384 elements/token
at f16                                           = 32 KiB/token

The other 24 layers hold a fixed-size recurrent state, sized by llama.cpp's n_embd_r() / n_embd_s() in src/llama-hparams.cpp, fed by the GGUF's ssm.conv_kernel = 4, ssm.inner_size = 4096, ssm.group_count = 16, ssm.state_size = 128:

n_embd_r = (4-1) x (4096 + 2 x 16 x 128)  =    24,576
n_embd_s = 128 x 4096                     =   524,288
(24,576 + 524,288) x 24 layers x 4 bytes  = 50.25 MiB per sequence
x 4 sequences (llama-server's default)     = 201 MiB = 0.196 GiB

That last multiplication is easy to miss and it is not optional. src/llama-model.cpp sizes the recurrent state with /* recurrent_rs_size */ std::max((uint32_t) 1, cparams.n_seq_max) on the branch that names LLM_ARCH_QWEN35, cparams.n_seq_max comes from params.n_parallel, and tools/server/server.cpp resolves the server's auto default with params.n_parallel = 4 — so the allocation is four sequences' worth whether or not you use them. The KV cache is not multiplied: the same default also sets kv_unified = true, so the four slots share one context window rather than each getting their own. The recurrent term stays context-independent, which is what makes it easy to overlook.

The choice: quant, context, or both

32 KiB/token is a small KV term for a 262,144-context model, so context is unusually cheap here and quality is the expensive axis. Price the two moves against an 8 GB-class configuration (Q4_K_M at 32,768, which totals 4.509 GiB):

• stepping the quant Q4_K_M → Q8_0 costs 4.812 − 3.313 = 1.499 GiB
• doubling the context 32,768 → 65,536 costs 1.000 GiB

Together that is 2.499 GiB against the 4 GiB of extra capacity, so this is not an either/or. Take both — the result still leaves 4.992 GiB of the card unspent:

Quant (+ f16 mmproj)	Weights	KV @ 65,536	Recurrent	Total
Q4_K_M	3.313 GiB	2.000 GiB	0.196 GiB	5.509 GiB
Q5_K_M	3.714 GiB	2.000 GiB	0.196 GiB	5.910 GiB
Q6_K	4.051 GiB	2.000 GiB	0.196 GiB	6.247 GiB
Q8_0	4.812 GiB	2.000 GiB	0.196 GiB	7.008 GiB
bf16	8.472 GiB	2.000 GiB	0.196 GiB	10.668 GiB

Why the quant gets the headroom first. Fara's output is pixel coordinates — the card describes the training data as "Curated datasets for predicting actions and pixel coordinates from screenshots, with images, text, and bounding boxes." Quantisation error lands on exactly that faculty, and no published evaluation measures how much grounding accuracy a given tier costs. On an 8 GB card Q8_0 is something you trade context for; here it is affordable outright at 40% of the card, so the sensible move is to take the quantisation variable off the table rather than economise on the model's core competence.

Why not the unquantised bf16 GGUF. It arithmetically fits — 8.472 GiB of weights plus 1.196 GiB at 32,768 context is 9.668 GiB — but that leaves 2.332 GiB for CUDA context, compute buffers, the vision graph and the GPU-accelerated Chromium this agent drives on the same card. It is a legitimate option if you dedicate the GPU and drive the display from something else; it is not a safe default.

Why 65,536 and not more. The harness asks that you "Please ensure that context length is set to at least 15000 tokens and temperature to 0 for best results.", so 65,536 is more than four times the vendor's floor. Longer still fits, but the native maximum does not:

Context	Weights	KV (f16)	Recurrent	Sum
32,768	4.812 GiB	1.000 GiB	0.196 GiB	6.008 GiB
65,536	4.812 GiB	2.000 GiB	0.196 GiB	7.008 GiB
131,072	4.812 GiB	4.000 GiB	0.196 GiB	9.008 GiB
262,144 (native)	4.812 GiB	8.000 GiB	0.196 GiB	13.008 GiB ✗

Those last terms — CUDA context, compute buffers, the vision graph — are not in the tables; they are allocated at load time and this recipe does not derive them. Neither is the browser: Playwright launches a real Chromium on this machine, and if it renders on the GPU its process holds memory on the same 12 GB. Read the CUDA0 compute buffer size lines that llama-server prints on startup, watch nvidia-smi with the agent actually running, and report what you see.

One of those undisclosed terms is large, and this recipe deliberately makes it larger. Step 3 passes --flash-attn off (Troubleshooting explains why). With flash attention on, the attention score matrix never exists as a tensor. With it off, src/llama-graph.cpp takes its other branch and builds kq = ggml_mul_mat(ctx0, k, q), forced to GGML_PREC_F32 — an explicit n_kv × n_ubatch × n_head f32 tensor in the compute buffer. With this model's 16 attention heads (text_config.num_attention_heads) and llama.cpp's default -ub 512, the worst case the graph reservation has to cover at this recipe's context is:

65,536 (full attention cache) x 512 x 16 x 4 B  = 2.000 GiB

That is the same size as the entire f16 KV term, and it is the price of the flag. It is an upper bound at reservation time, not a measurement, and it scales linearly with -ub — -ub 256 halves it at some cost in prompt-processing speed — and with -c, so the 32,768 row of the table above costs 1.000 GiB here instead of 2.000. The card still has room: ~7.0 GiB of derived state plus ~2 GiB of this is just over 9 GiB of 12, and the honest reading is that the comfortable slack quoted above shrinks from about 5 GiB to about 3. Check it against the CUDA0 compute buffer size line rather than trusting the arithmetic.

How much context one screenshot costs

A CUA sends a screenshot every step, so this is the number that decides whether the context budget above is honest. It is not the 768 in the projector metadata — that value is derived from num_position_embeddings (√2304 × 16 = 768) as a reference for position-embedding interpolation, and llama.cpp treats this projector as dynamic-resolution: clip.cpp groups PROJECTOR_TYPE_QWEN3VL with the cases that compute n_patches as (nx / (patch_size*2)) * (ny / (patch_size*2)), and the --image-min-tokens / --image-max-tokens pair is documented as "only used by vision models with dynamic resolution".

The harness fixes the browser viewport at 1440×900 (fara15_agent.py: viewport_width: int = 1440, viewport_height: int = 900), matching the card's note that "The screen resolution Fara is most commonly trained with is" 1440×900. It then applies Qwen's smart_resize with patch_size=16, merge_size=2, so the factor is 32:

900 -> round(900/32)*32   = 896        1440 -> round(1440/32)*32 = 1440
(1440 / 16) / 2 = 90 / 2 = 45 across   (896 / 16) / 2 = 56 / 2 = 28 down
image tokens = 45 x 28                 = 1,260 tokens per screenshot

The harness keeps max_n_images: int = 3, so 3,780 tokens of any prompt are screenshots and the rest of the growth is text — thoughts, tool calls, observations. Its own conservative default image_token_estimate: int = 1500 sits just above the derived 1,260, which is a useful sanity check on the arithmetic. That bounded image history is the other reason 65,536 is generous rather than arbitrary: nothing in the loop grows the image share.

Installation

1. Build llama.cpp with CUDA 12.8+ for sm_120

This is the one step that genuinely differs on Blackwell. The RTX 5070 is compute capability 12.0 per NVIDIA's CUDA GPUs table, and llama.cpp's CUDA CMakeLists annotates that target as "Blackwell, needs CUDA v12.8, FP4 tensor cores" — and it appends 120a-real to its fallback architecture list only inside a CUDAToolkit_VERSION VERSION_GREATER_EQUAL "12.8" guard. Check the toolkit first:

nvcc --version    # must report release 12.8 or newer

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

Two details worth knowing rather than pasting:

• Pass 120 explicitly rather than trusting the default. Left alone, llama.cpp does not assemble a fixed architecture list at all: the same CMakeLists sets CMAKE_CUDA_ARCHITECTURES to native whenever GGML_NATIVE is on, and ggml/CMakeLists.txt defaults GGML_NATIVE ON. So the default targets whichever card happens to be attached at compile time, and an inadequate toolkit fails the build rather than quietly shipping a binary with no kernels for your card. Naming the architecture yourself is what makes the build reproducible on a machine that is not the one the GPU sits in — and it keeps the failure at configure time, where you can see it.
• You write 120, the build system compiles 120a. The same file rewrites any plain 12X architecture into its architecture-specific 12Xa form, because — in its own words — "the Blackwell FP4 tensor core instructions are not forwards compatible". You do not need to type the a, and you should not be surprised to see 120a-real in the CMake status line.

Build from a llama.cpp release at or after b10087 — that is the release the GGUF used here was quantised on, per the quantiser's card.

A note on NVFP4, since Blackwell is why people ask. The runtime side is not the gap: GGML_TYPE_NVFP4 is a first-class quant type in ggml/include/ggml.h, and ggml/src/ggml-cuda/mmq.cu selects a native FP4 matmul for it behind blackwell_mma_available(cc) — a gate this card passes. The gap is that there is nothing to feed it: of the 33 Fara1.5 repositories on the Hub exactly one is NVFP4-named, wizardeur/Fara1.5-4B-NVFP4, and it holds a single .gitattributes file and no weights. So build for 120 because that is your architecture; the 4-bit path on this page is a GGUF K-quant through llama.cpp's ordinary CUDA kernels.

2. Download the weights and the projector

Both files are required. The projector is a separate artifact — downloading only the text GGUF gives you a model that cannot see.

pip install -U huggingface_hub

hf download bartowski/Fara1.5-4B-GGUF \
  Fara1.5-4B-Q8_0.gguf mmproj-Fara1.5-4B-f16.gguf \
  --local-dir ~/models/fara1.5-4b

Microsoft publishes no first-party GGUF for Fara1.5 — the microsoft org ships safetensors for all three sizes (and an ONNX repo only for the previous-generation Fara-7B). Four independent community quantisers have converted the 4B including its vision tower — bartowski, prithivMLmods, DevQuasar and runanywhere all ship an mmproj file. Four independent conversions succeeding is the strongest available evidence that the vision path converts cleanly; it is still not a runtime test.

3. Serve it

./build/bin/llama-server \
  -m ~/models/fara1.5-4b/Fara1.5-4B-Q8_0.gguf \
  --mmproj ~/models/fara1.5-4b/mmproj-Fara1.5-4B-f16.gguf \
  -ngl 99 -c 65536 --flash-attn off --port 5000 \
  --image-min-tokens 1024 \
  --reasoning-format none \
  --temp 0

Four of those flags are load-bearing and worth understanding rather than pasting:

• --flash-attn off. The Blackwell-specific one, and it is spelled out rather than left alone because the default is the risky setting. common/arg.cpp declares the flag as [on|off|auto] and fills its help text from the struct default, which common/common.h sets to LLAMA_FLASH_ATTN_TYPE_AUTO. auto does not mean "off unless asked for": src/llama-context.cpp computes cparams.flash_attn = params.flash_attn_type != LLAMA_FLASH_ATTN_TYPE_DISABLED, so auto starts enabled and is only downgraded if a support probe finds the fused attention node landing on a different device than its layer. On a CUDA build for this card it is not downgraded — it resolves to enabled, and the startup log says which. Flash attention on is the single strongest trigger in #25717's crash matrix: the only variable there that crashed on its own, with a nineteen-token prompt. The flag is not free — see the compute-buffer term derived in the budget above — and it does not make you immune, which Troubleshooting is explicit about. What it does is remove the one condition in that matrix which failed unconditionally.
• --image-min-tokens 1024. The projector ships no minimum-pixel key of its own — its 30 metadata entries cover geometry, normalisation and the projector type, nothing about a token floor — so llama.cpp falls back to its Qwen-VL default and prints a warning: "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks". Fara is a grounding model, so this is precisely the case the warning is about. At the harness's 1440×900 viewport the image is already 1,260 tokens and the floor never binds; it protects you if you shrink the viewport.
• --reasoning-format none. Under llama-server's default reasoning extraction, <think> content is pulled out into a separate reasoning_content field. Fara's chat template opens every assistant turn with <think> and never closes it, and the harness parses the raw text itself — it splits on <tool_call> and feeds the preceding text back as the agent's thoughts. Without this flag message.content comes back empty and the agent loop dies on round one. none is documented as "none: leaves thoughts unparsed in message.content", which is exactly what the harness expects.
• -c 65536. See the budget tables. Do not leave it at the model's native 262,144 — that alone needs 8 GiB of KV.

And one flag deliberately not added: --kv-unified. Both surviving configurations in #25717 ran a unified KV cache, so it is tempting to spell it out. On llama-server it is already the default and adding it changes nothing: common/arg.cpp sets params.n_parallel = -1 (auto) for the server example, and tools/server/server.cpp turns that into n_parallel = 4 and kv_unified = true. The instruction that matters is the inverse: do not add --parallel. Any explicit value leaves kv_unified at its false default and hands you the split cache that #25717's crashing rows ran — and --parallel 4 also divides your window, because llama-context.cpp sets n_ctx_seq = n_ctx only for a unified cache and n_ctx / n_seq_max otherwise. With -c 65536 that would give each slot 16,384 tokens: barely over the harness's 15,000-token floor, and none of the headroom this whole page is about.

4. Prove the vision path works before going further

First read three lines of the startup log: the CUDA0 device line (it must name your 5070 — if the model loaded on CPU, go back to step 1), the flash_attn line (it must report disabled; if it does not, your flag did not take), and the mmproj load stage.

Then check the server advertises multimodal and send a real image. The server README documents image_url as a first-class content part: "can be a remote URL, base64 (raw or URI-encoded via data:image/...;base64) or path to local file".

Test A — is the projector wired up at all?

curl -s http://localhost:5000/v1/models | grep -o multimodal

curl -s http://localhost:5000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "fara",
  "temperature": 0,
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "Describe this image in one sentence."},
    {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}
  ]}]
}' | python3 -m json.tool

If the reply describes the picture, the projector is wired up. If it answers as though no image were present, or the server never logged an mmproj load, stop here — the agent loop below will silently produce nonsense actions rather than fail loudly. On a freshly-released architecture this step earns its keep twice over: it separates "my build has no kernels for this card" from "the model is bad at this".

Test B — and on Blackwell this is the one that earns its keep. Test A puts only a handful of text tokens in front of the image, the same shape as #25717's short-prompt rows (~19 tokens) — which is exactly what that report describes as passing once flash attention is off. The shape it describes as still crashing is a long text prefix, roughly 800 tokens, ahead of the image, and a real agent prompt is the long one. So probe it before you trust the setup:

python3 - <<'PY'
import json, urllib.request
long_text = "Describe precisely. " * 200          # ~800 tokens of text, BEFORE the image
body = {"model": "fara", "temperature": 0, "max_tokens": 128,
        "messages": [{"role": "user", "content": [
            {"type": "text", "text": long_text},
            {"type": "image_url", "image_url": {"url":
                "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}]}]}
req = urllib.request.Request("http://localhost:5000/v1/chat/completions",
                            data=json.dumps(body).encode(),
                            headers={"Content-Type": "application/json"})
print(urllib.request.urlopen(req, timeout=600).read().decode()[:400])
PY

--image-min-tokens 1024 floors even that small logo at 1,024 image tokens, so the image batch is a realistic size rather than a toy. If Test B returns a description you have cleared both reported triggers, on a build newer than any in the report. If Test B kills the server while Test A passed, you have reproduced #25717 on a new model — look for find_slot: non-consecutive token position above the CUDA error in the log, read the Troubleshooting entry, and please post it upstream.

5. Install the harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Running

Two client paths share the same llama-server endpoint. Path A (fara-cli) is the smoke test — unsandboxed. Path B (Magentic-UI) is what the vendor recommends for anything real. Pick deliberately; they are not equivalent in risk.

Path A — fara-cli, for a first smoke test

fara-cli speaks to any OpenAI-compatible endpoint. Point it at llama-server:

fara-cli \
  --base_url http://localhost:5000/v1 \
  --api_key sk-no-key-required \
  --model fara \
  --task "how many pages does wikipedia have" \
  --headful \
  --output_folder ./runs/first

--headful is not optional here, despite reading like a convenience flag. The harness runs the browser headless by default: run_fara.py declares the flag as "Run the browser in headful mode (show GUI, default is headless)" and passes headless=not args.headful into the browser launch. Omit it and a real Chromium still opens, still clicks and still submits — you simply cannot see it. Every supervision instruction in this section depends on the window being visible, so the flag is what makes the rest of the advice true rather than aspirational.

Each run writes "per-step screenshots and a data_point.json (task, actions, observations, and outcome)" to --output_folder. Watch that folder on the first few runs — it is how you tell a grounding failure (clicks landing in the wrong place) from a reasoning failure.

With that flag the command opens a real, unsandboxed Chromium on your desktop — the playwright install in step 5 is what put it there. Keep the task on a low-stakes public site, stay signed out, and stay at the keyboard. Fara is trained to stop and ask before entering personal information, making payments, submitting forms, signing in or sending messages; if you see it sail through one of those, stop the run — that is a wiring problem in your prompt path, not a model quirk you should tolerate.

Path B — Magentic-UI, for the container

Everything the model card asks for — the containerised browser, domain allow-lists, watch-mode and a pause control — lives in Magentic-UI, not in fara-cli. This is where a Docker requirement would come from; there is no Docker step in Path A because Path A genuinely does not use one.

Magentic-UI takes the same OpenAI-compatible endpoint you started above, so nothing on this page changes except the client. Its setup is documented in its own repository rather than reproduced here, because this recipe has not verified those steps end to end on this GPU — follow microsoft/magentic-ui and give it http://localhost:5000/v1 as the model endpoint.

Results

• Speed: omitted — no published measurement exists for Fara1.5-4B on this card, or on any consumer GPU. The only hardware the vendor names is datacenter class, and borrowing a number from it would be misleading for a 12 GB Blackwell card. Blackwell is also the architecture with the least community data for this model family: no NVFP4 build of this model exists to exercise the card's native FP4 path (see the NVFP4 note above), and the only published measurement of this architecture family's SSM kernels was taken on much older silicon (see Troubleshooting). Note too that Step 3 deliberately runs with flash attention disabled, which costs prompt-processing throughput — so if you do contribute a number, say whether you measured it with the flag as documented or with flash attention on. If you run this, please contribute your numbers so this section can be replaced with a measurement.
• VRAM usage: 4.812 GiB of weights and projector; 7.008 GiB including KV and the four-sequence recurrent state at 65,536 context, derived above from the GGUF's own metadata and llama.cpp's memory formulas. This is a derivation, not an observed peak — compute buffers, the vision graph and the browser's own GPU memory are additional. See /check/fara1-5-4b/rtx-5070.
• Quality notes: the vendor's published scores (80.8 WebVoyager, 57.3 Online-Mind2Web, 27.4 WebTailBench) are for the BF16 model on datacenter hardware. Q8_0 is the closest quantised tier to those weights, which is the whole reason it is the lead here; if you drop to Q4_K_M to free VRAM, you are trading against grounding accuracy on a model whose main job is predicting exact pixel coordinates, and no published evaluation measures that trade.
• Language: English only, per the model card's out-of-scope list.

For the full benchmark data, see /check/fara1-5-4b/rtx-5070.

Troubleshooting

The build succeeds but the GPU is barely used, or the server falls back to CPU

Suspect the toolkit before anything else, and check it directly: nvcc --version must report 12.8 or newer. Blackwell's FP4 tensor-core instructions are not forwards-compatible, so an older toolkit cannot emit sm_120a code at all.

A binary that is silently free of device code for your card can only come from llama.cpp's fallback architecture list — the else() branch of if (NOT DEFINED CMAKE_CUDA_ARCHITECTURES), which appends 120a-real only inside a CUDAToolkit_VERSION VERSION_GREATER_EQUAL "12.8" guard. That branch is reached only when GGML_NATIVE is off or CMake is too old; at its default the file sets CMAKE_CUDA_ARCHITECTURES to native instead, and step 1's explicit -DCMAKE_CUDA_ARCHITECTURES=120 skips the whole block either way. On this recipe's build, in other words, a too-old toolkit fails loudly rather than quietly. Do not try to confirm the toolkit from the configure log. The 12X → 12Xa rewrite in that same CMakeLists is unconditional, so 120a appears in the resolved list on every toolkit version and proves nothing. If the GPU really is idle with a build that configured and compiled, look past CUDA to -ngl and to whether llama-server logged the CUDA device at startup.

CUDA error: an illegal memory access was encountered on a vision request

This is the Blackwell failure worth knowing about, and it is why Step 3 disables flash attention and Step 4 has two tests. llama.cpp #25717 reports llama-server dying with exactly that message on vision requests, preceded in the log by find_slot: non-consecutive token position 774 after 773 for sequence 0 with 2040 new tokens. The reporter bisected seven configurations on an RTX 5070 Ti 16 GB (Blackwell, sm_120), driver 610.74 — one tier up from this card, same silicon. Two of the seven completed: flash attention off with the default unified cache and a short text prefix, and one where the same long prompt was sent with the image as the first content part. Every row with flash attention on crashed, including one with a nineteen-token prompt. His reading of the mechanism is that Qwen3.5's M-RoPE 3D position ids are mishandled on the flash-attention path, and that the position bookkeeping requires consecutive positions an image batch violates.

What transfers, precisely. The report is about a different model, and the honest summary is same machinery, different model, unverified outcome. On the transfer side:

• Fara1.5-4B genuinely uses M-RoPE. Its config.json carries rope_parameters.mrope_section = [11, 11, 10] with mrope_interleaved: true, and the GGUF's qwen35.rope.dimension_sections reads [11, 11, 10, 0] — the same section layout the reported model's GGUF carries.
• Both load a qwen3vl_merger projector at patch_size = 16, spatial_merge_size = 2. Same projector implementation; only the tower's depth and width differ.
• The warning that precedes the crash is emitted by src/llama-memory-recurrent.cpp — the recurrent half of the hybrid cache. Its find_slot compares the last position in a batch against cell.pos + n_seq_tokens and warns when the two disagree, under the frank source comment "What should happen when the pos backtracks or skips a value?". That check exists because the architecture is hybrid, which is precisely why 24 of this model's 32 layers pass through it. It is neither MoE-specific nor size-specific.

Against the transfer:

• The reported model registers as qwen35moe — a different llama.cpp architecture from this model's qwen35 — with 40 layers to 32 and 2 KV heads to 4.
• Every row of the matrix ran 24 layers' experts on the CPU (--n-cpu-moe 24) with -b 4096 -ub 4096. A fully-offloaded 4B at llama.cpp's defaults (-b 2048 -ub 512) is a different placement and a different batch split — and the reporter himself lists #19929, "batch-size dependent M-RoPE weirdness", as related.
• It is one report from one person with no maintainer reply. The thread's only comment is his own follow-up adding reproducers; the only other timeline activity is a different user cross-referencing an unrelated Qwen3.6 bug. No pull request references it, so nothing has been fixed — and nothing has been ruled out either. Its newest build, b10016, is described in the report as "current latest" on 15 July, older than the b10087 floor this recipe builds from, and nobody has re-tested since.

What this recipe does about it. --flash-attn off removes the one condition that crashed unconditionally, and Test B probes the long-prefix trigger before you commit a run. This is mitigation, not immunity: in the report, flash attention off still crashed once ~800 text tokens preceded the image, and adding --kv-unified did not rescue that case. One piece of luck is on your side — the harness puts the screenshot first in every user turn and the text after it (content=[ImageObj.from_pil(scaled_screenshot), text_prompt] in fara15_agent.py; the opening turn has the same shape with the task instruction), which is the ordering of the row that completed. But Fara's system prompt is long text and still precedes that image in the flattened sequence, and — as the context section notes — the text history grows every round while the image history stays capped, so the prefix in front of each screenshot only gets longer. Test B is your detector; the first agent run is the experiment.

There is one earlier Blackwell precedent, worth reading for the variables it names rather than as corroboration: #21564, an Xid 43 illegal-memory-access on an RTX 5090 that a bisect pinned to a flash-attention kernel optimisation. It closed with nobody else having reproduced it — an NVIDIA engineer could not on a newer driver and CUDA toolkit, and a maintainer remarked that the report itself looked machine-written. Its durable lesson is that driver and toolkit version were the axis that moved the outcome, so record both if you end up filing anything.

ollama run fara does nothing — there is no Ollama entry

There is no Ollama library entry for Fara1.5 in any form: fara1.5, fara, fara1-5 and fara-7b all 404. The GGUF/LM Studio/Ollama line in the harness README refers to Fara-7B, the previous generation, reached with a --fara-7b flag — it is not about Fara1.5, and a lead aggregator has already mis-attributed it. Likewise, the "run them in your choice of tools" list on the quantiser's card is that quantiser's standard boilerplate, printed with the caveat that "if it's a newly supported model, you may need to wait for an update from the developers"; it is not a statement that LM Studio or Jan has tested this model. llama.cpp built from source is the path this recipe verifies.

The model answers but ignores the screenshot

Almost always the projector. Confirm --mmproj is on the command line, that the server logged an mmproj load stage, and that Step 4's curl describes the image. A text-only GGUF paired with no projector loads and runs perfectly happily — it simply cannot see, which in a CUA presents as confidently wrong actions rather than an error.

Clicks land near targets but not on them

Check the viewport first. The harness defaults to 1440×900 because that is what the model was mostly trained at; a different window size changes both the grounding distribution and the token cost derived above. Then confirm --image-min-tokens 1024 is set, since llama.cpp's own warning ties sub-1024-token images to degraded grounding.

If both are right, there is one more thing worth ruling out before you blame the quantisation. A community user reported that llama-server degrades fine-grained vision accuracy for PROJECTOR_TYPE_QWEN3VL models relative to llama-cli on the same build, with identical prompt-token counts — llama.cpp issue #22785, filed against the Vulkan backend at release b8545. Read it with three caveats: a maintainer disputed the reporter's bisect and asked for a proper one, the issue was auto-closed as stale and unconfirmed, and nobody re-tested it on a CUDA build or anywhere near b10087. Its value here is as a diagnostic, not a known defect: push the same screenshot through llama-cli and compare what the model says about it. If the CLI reads the page correctly and the server does not, you have reproduced something real and worth reporting upstream.

rope.dimension_sections has wrong array length; expected 4, got 3

This is a real open llama.cpp bug (PR #25334), but it belongs to Ollama-exported qwen35 blobs, not to the GGUF this recipe uses. The Q8_0 artifact's own metadata reads qwen35.rope.dimension_sections = [11, 11, 10, 0] — four entries — so it cannot hit this failure. If you see this error, you are loading a different file than the one downloaded in Step 2.

Generation is slower than a 4B model should be

There is an open, community-reported performance regression affecting exactly the SSM kernels this architecture leans on — llama.cpp issue #25162 reports a 24-42% loss on an RTX 2080 Ti (Turing, SM75) after commit 9e58d4d69, and notes the Qwen35 architecture uses linear-attention layers for about 75% of its blocks (which matches the 24-of-32 split derived above). Three caveats before you act on it: it has no maintainer response; it was measured only on Turing, several architecture generations before this card's Blackwell (SM120), and nobody has posted numbers for any Blackwell part; and the one candidate fix raised in the thread, PR #25185, was merged on 2026-07-01 with a stated scope of flash-attention kernels rather than the SSM path, with no re-measurement posted since. Release b10087 was published on 2026-07-22, so a build from it or newer already contains that merge — though with the --flash-attn off Step 3 sets, a fix scoped to flash-attention kernels cannot be doing anything for you either way. Treat the regression as a lead to check with llama-bench across a couple of builds, not as a known property of this card.

Out of memory at load

Do not reach for the usual --cache-type-k q8_0 --cache-type-v q8_0 here. With the --flash-attn off this recipe needs, a quantised V cache is not a slow path or a warning — it is a refusal to start. src/llama-context.cpp rejects the pair outright, logging "quantized V cache requires flash_attn to be enabled" and returning a null context; a second guard inside the constructor throws "quantized V cache was requested, but this requires Flash Attention". It is the most-copied OOM remedy on the internet and it is the one thing you must not paste here.

Quantising the K half alone is legal without flash attention, and the dimension gate passes: ggml's block_q8_0 is one ggml_half scale plus 32 int8_t quants — 34 bytes per 32 values against f16's 64 — and n_embd_head_k of 256 divides that block size cleanly. Per token, across the 8 full-attention layers:

K: 8 x 4 x 256 = 8,192 elems -> f16 16.0 KiB  ->  q8_0 (8,192/32) x 34 B = 8.5 KiB
V: 8 x 4 x 256 = 8,192 elems -> f16 16.0 KiB  ->  unchanged            = 16.0 KiB
total per token                    32.0 KiB   ->                         24.5 KiB

so --cache-type-k q8_0 takes the KV term from 2.000 GiB to 1.531 GiB at this recipe's 65,536 context, and the whole derived total from 7.008 to 6.539 GiB. In descending order of what the levers actually buy you on this card:

Lever	Frees	Costs you
-c 32768	1.000 GiB	half the window; still double the harness's 15,000-token floor
-ub 256	~1.000 GiB	prompt-processing throughput — it halves the flash-attention-off score matrix
--no-mmproj-offload	0.626 GiB	slower image encoding, once per agent turn, on the CPU
--cache-type-k q8_0	0.469 GiB	a little K-cache precision; needs no flash attention

Keep the f16 projector rather than swapping in a q8_0 one: at 0.626 GiB it is not where the savings are, and it trades vision fidelity in a model whose entire job is looking at pixels — if you need it out of VRAM, move it to the CPU with --no-mmproj-offload instead of quantising it. And if you are already at a comfortable margin and still OOM, check what else holds VRAM: on a card that also drives your display, the browser this agent opens is competing for the same 12 GB.