Fara1.5-4B on RTX 5080: bf16 Weights and mmproj Vision, No Quantisation Step

What You'll Build

A local browser computer-use agent (CUA): Microsoft's Fara1.5-4B served by llama-server on an RTX 5080, with the vision projector loaded so the model actually sees browser screenshots, driven by Microsoft's own fara-cli harness against a plain local Playwright Chromium.

For a 4B model this card inverts the usual question. On 8 GB the recipe is forced into a 4-bit quant; on 12 GB it can afford Q8_0; on 16 GB the quantisation step disappears entirely — the unquantised bf16 weights fit at a 65,536-token context with room unspent even after the flash-attention-off compute buffer derived below, so this page serves the model exactly as Microsoft evaluates and serves it. More VRAM than this would only buy context the harness cannot fill, so as far as configuration goes, 16 GB is the ceiling for Fara1.5-4B and this card is at it. What separates it from the other 16 GB Blackwell parts is therefore throughput rather than anything on this page, and nobody has measured that for this model (see Results).

Hardware data: RTX 5080 (16GB VRAM) · bf16 + f16 mmproj = 8.472 GiB of weights, 10.668 GiB accountable at 65,536 context (derived below) · See benchmark data

⚠️ This model clicks, types and submits in a real browser, and the path documented here gives it no isolation. That is the operating requirement, not boilerplate, and it is the thing to settle before anything else on this page.

The model card is blunt about the consequence: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." That sentence closes the card's Recommended Deployment: MagenticLite section, which lists four controls as MagenticLite features — the first being "the browser runs in a Docker container with no access to host files or environment variables" — and then says: "If you integrate Fara1.5-4B directly, you're responsible for these controls."

fara-cli is a direct integration. Its install step is playwright install, and it drives a plain local Playwright Chromium — no container, no allow-list, no pause control. The harness README offers the containerised option as an alternative to it rather than as part of it: "instead of fara-cli you can use Magentic-UI", described in the same sentence as "a sandboxed browser environment with auditable action logging and user prompts at critical points". For the CLI path the vendor's position is a recommendation, not something shipped: the README's own note reads "We recommend running it in a sandboxed environment, monitoring its execution, and avoiding sensitive data or high-risk domains."

So on the fara-cli path you are the isolation boundary. Practically: a throwaway OS user or a VM, Playwright's own fresh browser profile (never your daily one), signed out of everything, low-stakes public sites, and eyes on the run rather than leaving it unattended. If you want the container, the allow-lists, watch-mode and the pause control, use Magentic-UI as the client instead — see Running.

ℹ️ Vision is the product, not a bonus. Per the model card, "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree." A Fara install that only serves text is a broken install — the --mmproj step below is mandatory, and Step 5 exists to prove that an image really reached the model before you let it drive anything.

⚠️ Runtime status — read before you commit an evening. llama.cpp support for this architecture is verifiable statically and this recipe shows that evidence rather than asserting it. What does not exist anywhere public is a report of Fara1.5 decoding an image under llama-server on any GPU. Two open Blackwell reports shape the commands on this page instead: #23385, whose hardware line reads "RTX 5090 and RTX 5080 (Blackwell SM 12.0)" and which describes a fatal abort in the quantised matmul kernels caused by an early-driver bug; and #25717, a CUDA illegal-memory-access on vision requests for a larger model of this architecture family, filed on a 16 GB Blackwell card. Neither is about Fara1.5, and Troubleshooting is explicit about what transfers — including how little the flash-attention flag in Step 3 actually guarantees. Treat your first run as the test, and see Results for why there is no speed figure here either.

Requirements

Component	Minimum	This recipe
GPU	16GB VRAM, CUDA compute 12.0 (sm_120)	— not measured; the budget below is derived (contribute a run)
RAM	16GB, 32GB comfortable — see the note below	—
Storage	9.10 GB (bf16 + f16 mmproj)	9,096,817,152 B = 8.472 GiB
Software	CUDA Toolkit 12.8+, llama.cpp b10087+, Python 3.10+, Playwright	—

Microsoft ships safetensors only — the two shards total 8.455 GiB per the HF tree API. Its Quickstart serves those with vLLM and names datacenter hardware for the job: "A GPU with enough memory for a 4B model in bf16 (A6000, A100, H100, and B200 have been tested)", and the card's highlights claim only that the model "runs on a single A100/H100/B200 with room for the screenshot history". The consumer route is the GGUF one documented here — but note the dtype: the vendor's reference invocation is vllm serve microsoft/Fara1.5-4B --dtype bfloat16. bf16 is what Microsoft runs, and it is what fits here.

On the RAM row. The weights are mmap'd, so most of that 9.1 GB is file-backed rather than resident. The reason to want 32 GB is llama-server's prompt cache: its default ceiling is 8 GiB of system RAM (cache_ram_mib = 8192 in common/common.h), and a CUA sending a distinct screenshot every step is the workload that fills it. On a 16 GB machine cap it with --cache-ram; see Troubleshooting.

What 16 GB actually buys — the budget, derived

There is no benchmark for this pair yet — /check reports unknown — so every number below is arithmetic from the artifact's own metadata and llama.cpp's source, not a measurement. Reproduce it rather than trusting it. All totals are GiB (2^30 bytes), the unit a "16GB" NVIDIA card is really in, so they compare directly against 16.

Weights. Byte counts from the bartowski GGUF tree:

Component	Bytes	GiB
Fara1.5-4B-bf16.gguf	8,424,393,568	7.846
mmproj-Fara1.5-4B-f16.gguf	672,423,584	0.626
Weights total	9,096,817,152	8.472

KV cache — and this is where Fara1.5 is unusual. Its qwen35 architecture is a hybrid: llm_arch_is_hybrid() in src/llama-arch.cpp returns true for LLM_ARCH_QWEN35, and src/models/qwen35.cpp marks layers recurrent off the full_attention_interval. The bf16 file's own header carries qwen35.block_count = 32 and qwen35.full_attention_interval = 4, and the source config.json spells the split out layer by layer in layer_types: 24 of 32 layers are linear_attention, only 8 are full_attention. Only those 8 carry a cache that grows with context.

With qwen35.attention.head_count_kv = 4 and attention.key_length = attention.value_length = 256:

K+V per token per attention layer = 2 x 4 x 256 x 2 B (f16) = 4,096 B
x 8 full-attention layers                                   = 32 KiB / token

A non-hybrid 32-layer model with the same head geometry would cost four times that. Context on this model is cheap — which is exactly why the weights, not the window, are what a 16 GB card should spend on.

Recurrent state — fixed, and tiny. The other 24 layers hold a context-independent state, sized by n_embd_r() / n_embd_s() in src/llama-hparams.cpp from the artifact's own GGUF keys (ssm.conv_kernel 4, ssm.inner_size 4096, ssm.group_count 16, ssm.state_size 128):

conv state : (4-1) x (4096 + 2 x 16 x 128) =  24,576 elems
ssm  state :         128 x 4096            = 524,288 elems
per layer  : 548,864 x 4 B (F32)           = 2.094 MiB
x 24 layers                                = 50.25 MiB per sequence
x 4 sequences                              = 201 MiB = 0.196 GiB   <- does not grow with context

That last multiplication is easy to miss and it is not optional. src/llama-model.cpp sizes the recurrent state with /* recurrent_rs_size */ std::max((uint32_t) 1, cparams.n_seq_max), and tools/server/server.cpp resolves the server's auto default with params.n_parallel = 4, so a plain llama-server reserves four sequences' worth whether or not you use them. The KV cache is not multiplied the same way: that same default also sets params.kv_unified = true, so the 32 KiB/token derived above is the whole KV cost regardless of slot count.

The whole ladder, in one table. Accountable allocations (weights + projector + KV + recurrent state) in GiB, each cell summed from exact byte counts and rounded once:

Context	Q4_K_M	Q5_K_M	Q6_K	Q8_0	bf16
32,768	4.509	4.910	5.247	6.008	9.668
65,536	5.509	5.910	6.247	7.008	10.668
131,072	7.509	7.910	8.247	9.008	12.668
262,144 (native)	11.509	11.910	12.247	13.008	16.668 ✗

Against a 12 GB card exactly two cells change status, and it is worth separating the one worth having from the one that is not:

• bf16 becomes affordable, with room. On 12 GB the unquantised weights reach only 32,768 context (9.668 GiB) and leave under 2.5 GiB for everything else, which is why the 12 GB recipes decline them. Here bf16 at 65,536 is 10.668 GiB and leaves 5.332 GiB, or 3.332 GiB once the flash-attention-off compute buffer derived further down is budgeted. Compare it against the right sibling. The Blackwell 12 GB recipe passes the same --flash-attn off and pays the same 2.000 GiB, so its Q8_0 lead comes down from 4.992 GiB of slack to about 3.0 — which means this card runs the unquantised weights with slightly more room than the 12 GB Blackwell card has for Q8_0. The Ada and Ampere 12 GB recipes are the ones that carry no such buffer, because they leave flash attention at its default; their slack figures are not comparable with these. 131,072 also fits, at 12.668 GiB, if your trajectories run long — but that leaves 3.332 GiB before the buffer, so pair it with a lower -ub.
• The native 262,144 window becomes reachable at Q8_0 (13.008 GiB) — and it is not worth buying. The harness retains at most three screenshots (below), and the README asks for a floor rather than a ceiling: "Please ensure that context length is set to at least 15000 tokens and temperature to 0 for best results." Nothing in the loop fills a quarter-million tokens, so that 8 GiB of KV would buy an empty window at the price of the weights.

One cell still does not fit at 16 GiB — bf16 at the native window, 16.668 GiB. Cap -c whichever tier you run.

Why the weights get the money. Fara's output is pixel coordinates — the card describes the training data as "Curated datasets for predicting actions and pixel coordinates from screenshots, with images, text, and bounding boxes." Quantisation error lands on exactly that faculty, and no published evaluation measures how much grounding accuracy any tier costs. The vendor's published scores are for bf16. On this card the trade is simply not necessary, and removing an unmeasured variable beats buying a window nothing will fill.

And a Blackwell-specific reason that does not apply to the Ada or Ampere cards in this class. llama.cpp's quantised matrix-multiply path is called MMQ, and it is the subject of an open bug that names this exact GPU: #23385, "Fatal MMQ crashes on Blackwell (RTX 5090/5080) due to unhandled sharedMemPerBlockOptin driver bug". A bf16 model never enters that path. ggml_cuda_should_use_mmq in mmq.cu only accepts quantised src0 types, so bf16 dispatches through mul_mat_vec_f for single-token decode and cuBLAS for batched work — both of which accept GGML_TYPE_BF16 natively in ggml-cuda.cu, so there is no dequantise-to-f16 detour either. Leading with unquantised weights removes a whole class of Blackwell failure rather than working around it. Troubleshooting covers what to do if you take the Q8_0 option and hit it anyway.

What bf16 costs, stated plainly. The bf16 file is 1.875× the size of the Q8_0 one (8,424,393,568 / 4,493,954,144), so every decoded token reads 1.875× as many weight bytes. Nobody has measured what that does to Fara's step latency on this card and this recipe does not guess (see Results). If you want the cheaper configuration, Q8_0 at 65,536 (7.008 GiB) runs under the identical command — swap the filename in Step 3. Choose Q8_0 if this card also drives your display and a GPU-composited browser, or if you want the faster agent loop; choose bf16 if you want the model as evaluated. Every 16 GB Blackwell card gets the identical memory budget from the table above, so this is a throughput decision, and this recipe has no measurement of it for either weight tier — which is exactly the gap the /contribute form exists to close.

At bf16 / 65,536 the 10.668 GiB above leaves 5.332 GiB of the card — and one of the terms that has to come out of it is large enough to name, because this recipe turns flash attention off (Step 3 explains why). The non-flash-attention branch of build_attn_mha in src/llama-graph.cpp materialises the attention-score tensor and pins it to F32 — ggml_mul_mat_set_prec(kq, GGML_PREC_F32), with the comment "this op tends to require high floating point range". That tensor is n_kv x n_tokens x n_head x 4 B, and the graph is reserved for the worst case, so with n_kv at the full 65,536, n_tokens at the default n_ubatch of 512 (common/common.h) and qwen35.attention.head_count = 16:

65,536 x 512 x 16 x 4 B = 2,147,483,648 B = 2.000 GiB   <- upper bound, prompt-processing only

Treat that as an upper bound on one tensor, not a measurement: the graph allocator may hold more than one buffer of that shape, and at decode time n_tokens is 1 so the term nearly vanishes. Budgeting it anyway leaves 3.332 GiB for the CUDA context, the vision encode graph, your desktop and the browser — comfortable, but not the 5.3 GiB a reader would infer from the table alone. -ub is the lever: -ub 256 halves the bound to 1.000 GiB at some prompt-processing throughput, and the flash-attention path would avoid it entirely if it were safe to use here. Turning flash attention off does not change the KV bytes themselves — attn_v_trans is set from !cparams.flash_attn in src/llama-model.cpp, which transposes V rather than resizing it.

The remaining terms are not in the table — they are allocated at load time and this recipe does not derive them. Read the CUDA0 compute buffer size lines llama-server prints on startup for the real figures on your build, watch nvidia-smi with the agent actually running, and report what you see.

How the context is actually spent

A CUA sends a screenshot every step, so the per-image cost decides whether the budget above is realistic. It is not the 768 in the projector metadata: that is clip.vision.image_size, a reference value, and llama.cpp treats this projector as dynamic-resolution — --image-min-tokens and --image-max-tokens are documented as "only used by vision models with dynamic resolution".

The harness fixes the browser viewport at 1440x900 (viewport_width: int = 1440, viewport_height: int = 900 in fara15_agent.py), matching the card's note that "The screen resolution Fara is most commonly trained with is" 1440x900. It pre-resizes with Qwen's smart_resize at PATCH_SIZE = 16 and MERGE_SIZE = 2, so the factor is 32 and 900 becomes 896. llama.cpp then counts merged patches per axis in tools/mtmd/clip.cpp, where PROJECTOR_TYPE_QWEN3VL returns (img->nx() / params.patch_size) / 2 across and (img->ny() / params.patch_size) / 2 down:

across : (1440 / 16) / 2 = 90 / 2 = 45
down   : ( 896 / 16) / 2 = 56 / 2 = 28
tokens : 45 x 28                 = 1,260 per screenshot
x max_n_images = 3               = 3,780 tokens resident

The harness's own conservative default is image_token_estimate: int = 1500, a useful sanity check on the arithmetic, and the model card agrees on the retention: "We only keep the most recent 3 screenshots" in the chat history.

So images are capped. What grows is the text trajectory: every round appends the model's reasoning, its tool call and an observation (max_observation_chars: int = 1000), and fara-cli defaults to --max_rounds 100. Microsoft's framing of the 262K window is "Long enough for multi-screenshot trajectories with full action history." On this card the window is not the scarce resource, so 65,536 is generous-and-cheap rather than a limit you will feel — and the growing text prefix in front of each screenshot is the variable Step 5's second test exists to probe.

Installation

1. Build llama.cpp with CUDA 12.8+ for sm_120

This is the one step that genuinely differs on Blackwell. The RTX 5080 is compute capability 12.0 — NVIDIA's CUDA GPUs table lists it in the same 12.0 group as the rest of the GeForce RTX 50 series — and llama.cpp's CUDA CMakeLists annotates that target as "Blackwell, needs CUDA v12.8, FP4 tensor cores". Check the toolkit first, because this is the only reliable way to check it:

nvcc --version    # must report release 12.8 or newer

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

Two details worth knowing rather than pasting:

• Pass 120 explicitly — but not because the default would silently miss your card. It would not. GGML_NATIVE defaults ON in ggml/CMakeLists.txt unless you are cross-compiling or have SOURCE_DATE_EPOCH set, and the CUDA CMakeLists then takes set(CMAKE_CUDA_ARCHITECTURES "native") before it ever reaches the toolkit-version list — so the default path targets whichever card is attached at compile time, and an inadequate toolkit fails the build loudly. What naming 120 buys is a build that does not depend on what was plugged in when you compiled, which matters in a container or on a separate build host, and it closes the one soft failure the native path has: llama.cpp's build guide notes that when nvcc cannot detect the GPU you get only a warning — "Cannot find valid GPU for '-arch=native', default arch is used" — and then a binary built for something else. The explicit form is what that guide's Override Compute Capability Specifications section documents.
• You write 120, the build system compiles 120a. The same file rewrites any plain 12X architecture into its architecture-specific 12Xa form because, in its own words, "the Blackwell FP4 tensor core instructions are not forwards compatible". You do not type the a, and you should not be surprised to see 120a-real in the CMake status line. Do not read that line as a toolkit check — Troubleshooting explains why it proves nothing.

Both halves of the model are first-class in mainline — worth confirming before you download 9 GB:

grep -n 'LLM_ARCH_QWEN35,' src/llama-arch.cpp        # -> { LLM_ARCH_QWEN35, "qwen35" }
grep -n 'qwen3vl_merger' tools/mtmd/clip-impl.h      # -> PROJECTOR_TYPE_QWEN3VL

Build from a llama.cpp release at or after b10087 — the release the GGUF used here was quantised on, per the quantiser's card.

A note on FP4, since this is the card people ask about it on. The FP4 tensor cores are real and llama.cpp does use them: GGML_TYPE_NVFP4 is a shipped quant type in ggml/include/ggml.h, and mmq.cu gates a native path on your architecture with use_native_fp4 = blackwell_mma_available(cc) && (src0->type == GGML_TYPE_MXFP4 || src0->type == GGML_TYPE_NVFP4). What is missing is not the loader — it is an artifact: there is no NVFP4 build of Fara1.5-4B to load. The only NVFP4-named repository on the Hub for this model holds a single file, .gitattributes, with no weights and no config. And FP4 is a 4-bit tier, i.e. the opposite direction from what this card's 16 GB is for; chasing it for a pixel-grounding model that already fits unquantised is backwards. Build for 120 because that is your architecture. (A separate and different proposal — PR #23572, whose author is explicit that "I intentionally avoided calling it NVFP4 because it is not currently compatible with NVFP4 checkpoints" — is open and unmerged, and is irrelevant here for the same reason.)

2. Download the weights and the projector

Both files are required. The projector is a separate artifact — downloading only the text GGUF gives you a model that cannot see.

pip install -U huggingface_hub

hf download bartowski/Fara1.5-4B-GGUF \
  Fara1.5-4B-bf16.gguf mmproj-Fara1.5-4B-f16.gguf \
  --local-dir ~/models/fara1.5-4b

Microsoft publishes no first-party GGUF for Fara1.5 — the microsoft org ships safetensors for all three sizes. Four independent community quantisers have converted the 4B including its vision tower: bartowski, prithivMLmods, DevQuasar and runanywhere each publish a matching mmproj. Four independent conversions succeeding is the strongest available evidence that the vision path converts cleanly; it is still not a runtime test.

For the cheaper configuration, substitute Fara1.5-4B-Q8_0.gguf (4,493,954,144 B = 4.185 GiB) here and in Step 3. Nothing else on this page changes.

3. Serve it

./build/bin/llama-server \
  -m ~/models/fara1.5-4b/Fara1.5-4B-bf16.gguf \
  --mmproj ~/models/fara1.5-4b/mmproj-Fara1.5-4B-f16.gguf \
  -ngl 99 -c 65536 --flash-attn off \
  --host 127.0.0.1 --port 5000 \
  --image-min-tokens 1024 \
  --reasoning-format none \
  --temp 0

Five of those flags are load-bearing and none is a default:

• --flash-attn off — the Blackwell one, and the reason it is spelled out rather than left alone: the flag's documented default is auto, and auto resolves to enabled. src/llama-context.cpp sets cparams.flash_attn = params.flash_attn_type != LLAMA_FLASH_ATTN_TYPE_DISABLED, so both AUTO and ENABLED switch it on. Flash attention on is the strongest single trigger in #25717's crash matrix — the only variable that crashed on its own, with a short prompt.
• No --parallel, and no --kv-unified either. A unified KV cache is what you want — it is the layout of both configurations in #25717's matrix that completed, and it keeps the whole window available to one request, because src/llama-context.cpp sets n_ctx_seq = n_ctx when the cache is unified and divides -c by the slot count when it is not. You get it by saying nothing: common/arg.cpp sets params.n_parallel = -1 for the server example, and tools/server/server.cpp then resolves that with if (params.n_parallel < 0) { params.n_parallel = 4; params.kv_unified = true; }. Passing --kv-unified is a no-op. Passing --parallel N is worse than a no-op: it suppresses that auto-block, so kv_unified falls back to false and your window is divided — --parallel 4 leaves 16,384 tokens per request, barely over the harness's 15,000 floor, and --parallel 8 leaves 8,192, under it. --parallel 1 is also present in both of the flash-attention-off rows that crashed.
• --reasoning-format none — mandatory. Without it the agent loop silently receives nothing; Troubleshooting has the mechanism.
• --image-min-tokens 1024 — this projector ships no minimum-pixel key of its own (its 30 metadata entries cover geometry, normalisation and the projector type, nothing about a token floor), so llama.cpp falls back to its Qwen-VL default and warns: "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks". Fara is a grounding model, so that warning is aimed precisely at this case. At the harness's viewport the image is already 1,260 tokens and the floor never binds; it protects you if you shrink the viewport.
• -c 65536 — see the budget table. Do not leave it at the native 262,144: at bf16 that is 16.668 GiB and will not load.

4. Confirm what actually loaded

./build/bin/llama-server --version    # build number >= 10087

Three lines of the startup log are worth reading before going further: the CUDA0 device line (it should name your 5080 — if the model loaded on CPU, go back to Step 1), the flash_attn line (it must say disabled; if it does not, your flag did not take), and the mmproj load stage.

5. Prove the vision path works — twice, and the second time is the one that matters

Check that the server advertises multimodal support, then send a real image. The server docs make the first part a client-side requirement rather than an optional nicety: "Clients should check /models or /v1/models for the multimodal capability before a multimodal request." Run GET /props alongside it — it reports which modalities loaded rather than merely that some did, the more useful signal when a projector silently failed. For the image itself, image_url.url "can be a remote URL, base64 (raw or URI-encoded via data:image/...;base64) or path to local file".

curl -s http://127.0.0.1:5000/v1/models | grep -o multimodal

# supplementary — names the modality that actually loaded
curl -s http://127.0.0.1:5000/props | python3 -c "import sys,json; print(json.load(sys.stdin)['modalities'])"
# expect: {'vision': True}

# test A — short prompt, image first
curl -s http://127.0.0.1:5000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "fara",
  "temperature": 0,
  "messages": [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}},
    {"type": "text", "text": "Describe this image in one sentence."}
  ]}]
}' | python3 -m json.tool

Test A is not sufficient on a Blackwell card. In #25717's matrix a short prompt with flash attention off completed, while the same configuration with roughly 800 text tokens in front of the image did not. Test B is deliberately harsher than what the harness sends — Fara puts the screenshot before the text, which is the ordering that survived in the report, whereas test B puts the text first. Run it anyway: it is the cheapest way to find out whether that failure exists on your build at all, and Fara's own long system prompt still precedes the image in the flattened sequence.

# test B — ~800 tokens of text BEFORE the image, which is the agent-run shape
python3 - <<'PY'
import json, urllib.request
long_text = "Describe precisely. " * 200
body = {"model": "fara", "temperature": 0, "max_tokens": 200,
        "messages": [{"role": "user", "content": [
            {"type": "text", "text": long_text},
            {"type": "image_url", "image_url": {"url":
             "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}]}]}
req = urllib.request.Request("http://127.0.0.1:5000/v1/chat/completions",
        data=json.dumps(body).encode(), headers={"Content-Type": "application/json"})
print(urllib.request.urlopen(req, timeout=300).read()[:400])
PY

If both replies describe the picture, you have cleared the risk this page is most worried about. If test B kills the server — watch the log for a find_slot: non-consecutive token position line immediately before the CUDA error — go to Troubleshooting. That would be the reported failure reproduced on a new model and a newer build, which is worth a comment on the upstream issue.

6. Install the harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Skip the README's [vllm] extra — that path serves bf16 safetensors on datacenter cards and is not what this recipe uses.

Running

Two clients share the same llama-server endpoint. Path A (fara-cli) is the smoke test, with no isolation. Path B (Magentic-UI) is what the vendor recommends for anything real. They are not equivalent in risk; pick deliberately.

Path A — fara-cli, for a first smoke test

fara-cli speaks to any OpenAI-compatible endpoint. Point it at llama-server:

source .venv/bin/activate
fara-cli \
  --base_url http://127.0.0.1:5000/v1 \
  --api_key sk-no-key-required \
  --model fara \
  --task "find the current UTC time on time.is and tell me" \
  --headful \
  --output_folder ./runs/first \
  --save_screenshots \
  --max_rounds 15

--headful reads like a convenience flag and is not one here. The harness runs the browser headless by default: run_fara.py declares the flag as "Run the browser in headful mode (show GUI, default is headless)" and passes headless=not args.headful into the launch, and the README shows it as opt-in. Either way this drives a real, unisolated Chromium on your machine — the danger is identical with or without the flag. What the flag changes is whether you can see it, and every supervision instruction below assumes you can, so it is what makes the rest of this advice true rather than aspirational.

Each step writes a screenshot and a data_point.json trajectory into --output_folder — watch that folder early, because it is how you tell a grounding failure (clicks landing in the wrong place) from a reasoning failure.

Keep the task on a low-stakes public site and stay signed out. Fara is trained to stop and ask before entering personal information, making payments, submitting forms, signing in or sending messages; if you see it sail through one of those, stop the run — that is a wiring problem in your prompt path, not a quirk to tolerate.

Path B — Magentic-UI, for the container

Everything the model card assumes — the containerised browser, domain allow-lists, watch-mode and a pause control — lives in Magentic-UI, not in fara-cli. That is also where a Docker requirement would come from; there is no Docker step in Path A because Path A genuinely does not use one.

Magentic-UI takes the same OpenAI-compatible endpoint you started above, so nothing on this page changes except the client. Its setup is documented in its own repository rather than reproduced here, because this recipe has not verified those steps end to end on this GPU — give it http://127.0.0.1:5000/v1 as the model endpoint. Whichever client you use, match the trained resolution: the card is explicit — "Match this in your sandbox for the most reliable grounding."

Results

• Speed: omitted — no published measurement of Fara1.5-4B exists for this card or for any consumer GPU, and the only hardware Microsoft names is datacenter class. Borrowing a figure from an A100 for a 16 GB Blackwell card would be misleading, and the vendor publishes no throughput numbers to borrow in the first place. This is the frustrating gap on this particular card, because throughput is the only thing it has over its 16 GB siblings and nobody has quantified it for this model. If you run this, please contribute your numbers so this line can be replaced with a measurement — and if you run both weight tiers, the bf16-versus-Q8_0 step latency is the most useful pair of numbers anyone could add to this page.
• VRAM usage: 8.472 GiB of weights and projector; 10.668 GiB accountable including KV and the four-sequence recurrent state at 65,536 context, derived above from the GGUF's own metadata and llama.cpp's memory formulas. This is a derivation, not an observed peak. Compute buffers, the vision graph and the browser's own GPU memory sit on top — and because this recipe disables flash attention, one of those buffers has a derived upper bound of 2.000 GiB at the default -ub 512, which the budget section works through. Track the empirical picture at /check/fara1-5-4b/rtx-5080.
• Quality notes: the vendor's published scores (80.8 WebVoyager, 57.3 Online-Mind2Web, 27.4 WebTailBench) are for the bf16 model on datacenter hardware. This recipe serves those weights unquantised, so there is no quantisation gap between the published numbers and what you are running — the remaining gap is the harness and the web, not the arithmetic. Drop to Q8_0 for bandwidth and you are trading against grounding accuracy on a model whose main job is predicting exact pixel coordinates, with no published evaluation of that trade.
• Language: English only — the card puts "Languages other than English (training data is English-only)" out of scope.

For the full benchmark data, see /check/fara1-5-4b/rtx-5080.

Troubleshooting

A fatal abort in the MMQ kernels, or a CUDA invalid-value at kernel launch

This one names your card. llama.cpp #23385 — "Fatal MMQ crashes on Blackwell (RTX 5090/5080) due to unhandled sharedMemPerBlockOptin driver bug" — is open, and the mechanism is a driver defect rather than a llama.cpp one: early Blackwell drivers return a corrupted cudaDeviceProp.sharedMemPerBlockOptin, either 0 or an overflowed 0x100000001, and llama.cpp maps that value straight onto the shared-memory budget its quantised matmul kernels dispatch against. ggml-cuda.cu still assigns info.devices[id].smpbo = prop.sharedMemPerBlockOptin unguarded, though mmq.cu has since added a floor ("MMQ tiles require at least 48 KiB per-block shared memory; fall back to BLAS otherwise.") that turns the 0 case into a cuBLAS fallback rather than an abort. The exact signature quoted in the issue (mmq_x_best=0, mmq.cuh:4135) belongs to an older code shape; in current master the equivalent abort prints J_best= first. A later commenter reports the same abort on a different Blackwell card, so it is not unique to the 5080/5090 pair.

Three ways out, in order: update the graphics driver; rebuild with -DGGML_CUDA_FORCE_CUBLAS=ON, a real option in ggml/CMakeLists.txt described as "always use cuBLAS instead of mmq kernels"; or run a non-quantised model, which is what this recipe leads with — ggml_cuda_should_use_mmq only accepts quantised weight types, so bf16 never reaches that code. If you switched to Q8_0 for bandwidth and then hit this, that is the trade showing up.

The server dies on an image request with CUDA error: an illegal memory access was encountered

This is why Step 5 has two tests. llama.cpp #25717 reports llama-server crashing with exactly that error on vision requests, preceded in the log by find_slot: non-consecutive token position N after N-1. The reporter bisected seven configurations; the triggers are flash attention, a split KV cache, and — the surprising one — the amount of text placed before the image in the prompt. Two rows completed: flash attention off with the default unified cache and a short text prefix, and one where the same long prompt was sent with the image as the first content part. The reporter's own summary of the mechanism is that Qwen3.5's M-RoPE 3D position ids are mishandled on the flash-attention path, and that the split-KV path requires consecutive positions that an image batch violates.

Read it with its limits stated:

• The hardware is an RTX 5070 Ti 16 GB (Blackwell, sm_120) — a sibling of this card at the same VRAM tier and the same compute capability, not this card.
• It is a community report with no maintainer engagement. It has exactly one comment, and that comment is the reporter adding his own copy-paste reproducer.
• It is a different llama.cpp architecture, not merely a different size. Read out of the reported artifact's own GGUF header, unsloth/Qwen3.5-35B-A3B-GGUF registers as qwen35moe — a separate arch enum from Fara's qwen35 in src/llama-arch.cpp — with block_count = 40 against Fara's 32, attention.head_count_kv = 2 against Fara's 4, embedding_length = 2048 against 2560, and expert_count = 256 against none. Its projector is a different and larger file.
• The configuration is not this one either. Every row of that matrix ran --n-cpu-moe 24 --batch-size 4096 --ubatch-size 4096 on Windows builds b10002/b10016; this recipe is fully offloaded at the default -b 2048 -ub 512 on b10087 or newer, i.e. an eighth of the physical batch. The reporter's own "Related" line points at #19929, "llama-server Qwen3.5 vision behave weirdly on certain batchsize settings." — closed as completed, with a partial fix he notes was already in his builds. Batch size is a live variable in this family of reports and his was unusually large.
• The mechanism, however, transfers better than any of that suggests. The non-consecutive token position line that precedes every crash is emitted by src/llama-memory-recurrent.cpp — the recurrent half of the hybrid cache, not the attention half and nothing to do with experts. Fara routes 24 of its 32 layers through exactly that memory. And the M-RoPE positioning the reporter blames is present in this artifact: its own header carries qwen35.rope.dimension_sections = [11, 11, 10, 0] and the source config sets mrope_interleaved: true. So the right posture is neither "this will happen to you" nor "this is about a different model" — it is "the two ingredients are both present here, nobody has run the combination, so test it in Step 5".

What actually protects you here is the message ordering, not the flag. Row 7 is the only long-prompt survivor in the matrix, and what it did differently was put the image first. Fara's harness does exactly that, at both of its call sites — content=[ImageObj.from_pil(scaled_screenshot), task_instruction] for the opening turn and content=[ImageObj.from_pil(scaled_screenshot), text_prompt] for every round after it, in fara15_agent.py. So the client you are pointing at this server produces the surviving configuration by construction, and you did not have to do anything to get it. The residual doubt is that Fara's system prompt is long text and still precedes the image in the flattened token sequence, and that prefix grows every round — so the shape is close to row 7 rather than identical to it.

--flash-attn off is still in Step 3, but understand what it buys: it removes the condition that failed unconditionally (rows 1–3 are every flash-attention-on row, and all three crash), and nothing more. It is not protection — rows 5 and 6 crash with it off, at 774 tokens of text before the image. If you send Fara-shaped prompts through some other client that puts text first, do not read the flag as a guarantee.

Step 5's test B exists to probe that: it sends ~800 text tokens before the image, which is deliberately the reported crashing order rather than the order the harness uses. If it crashes and test A does not, you have reproduced the report on a new architecture and a newer build, which is worth a comment on the upstream issue.

The build succeeds but the GPU is idle, or the server falls back to CPU

Check the toolkit first — nvcc --version must report 12.8 or newer, because Blackwell's FP4 tensor-core instructions are not forwards-compatible and an older toolkit cannot emit sm_120a code at all. But note what a successful build already tells you: with -DCMAKE_CUDA_ARCHITECTURES=120 named explicitly, a toolkit that cannot target sm_120 fails the compile rather than producing a quiet no-op, so if Step 1 completed then the toolkit is probably not your problem. One command rules the class out; then look at -ngl, at whether llama-server logged the CUDA device at startup, and at the prebuilt-binary trap below.

Whatever you do, do not try to confirm the toolkit from the configure log. The 12X → 12Xa rewrite in the CUDA CMakeLists is unconditional — it sits outside the if (NOT DEFINED CMAKE_CUDA_ARCHITECTURES) block that holds the CUDAToolkit_VERSION VERSION_GREATER_EQUAL "12.8" gate, and Step 1 passes -DCMAKE_CUDA_ARCHITECTURES=120 explicitly, which skips that block entirely. So Replacing 120 … with 120a prints on every toolkit version and proves nothing. nvcc --version is the test.

Relatedly: build from source rather than reaching for a third-party "sm120 Blackwell" binary. #26205 is an open report of llama-server crashing silently before it starts listening on an sm_120 card under exactly that kind of prebuilt Windows build, with the official older-CUDA build giving 0% GPU utilisation on the same machine — the two failure modes this step exists to prevent, in one report.

The agent does nothing — every step logs an empty response or list index out of range

This is the failure everyone hits, and it is not a bug in your setup. Fara opens a <think> block and never closes it. Under llama-server's default reasoning extraction the parser therefore treats the whole output — reasoning and the <tool_call> block together — as reasoning, returns it in reasoning_content, and leaves message.content empty. The harness parses message.content for <tool_call>, finds nothing, and the loop dies on round one.

Launch with --reasoning-format none, documented as "none: leaves thoughts unparsed in message.content". The raw output then begins with a stray unclosed <think> tag, which is cosmetic.

Reported by a community user on the model family's HF discussions and acknowledged there by Microsoft — swhitehead (an org member) asked for it to be filed upstream, which became microsoft/fara#82, still open. It applies to the 4B for a checkable reason rather than by assumption: the 4B's own GGUF carries the same chat template, whose generation prompt emits the unclosed <think>.

The model answers but ignores the screenshot

Almost always the projector. Confirm --mmproj is on the command line, that the server logged an mmproj load stage, that /v1/models advertises the multimodal capability, and that Step 5's /props call reports vision: True. A text-only GGUF with no projector loads and runs perfectly happily — it simply cannot see, which in a CUA presents as confidently wrong actions rather than an error.

Clicks land in the wrong place

Check the viewport first: the harness defaults to 1440x900 because that is what the model was mostly trained at, and a different window size changes both the grounding distribution and the token cost derived above. Then confirm --image-min-tokens 1024 is set, since llama.cpp's own warning ties sub-1024-token images to degraded grounding. Only after both should you look at the weights — and on bf16 there is nothing above you to climb to, which is exactly why this configuration is a good place to debug grounding from.

rope.dimension_sections has wrong array length; expected 4, got 3

This is a real open llama.cpp bug (PR #25334), but it belongs to Ollama-exported qwen35 blobs, not to the GGUF this recipe uses. The bf16 artifact's own header reads qwen35.rope.dimension_sections = [11, 11, 10, 0] — four entries — so it cannot hit this failure. If you see this error you are loading a different file than the one downloaded in Step 2. Relatedly, ollama run fara does nothing: there is no Ollama library entry for Fara1.5 in any form, and the registry manifest API has none either.

A long agent run climbs in system RAM, or the server dies after many steps

llama-server's prompt cache and context checkpoints are sized for chat, not for a stream of unique screenshots: the defaults are cache_ram_mib = 8192 and n_ctx_checkpoints = 32 per common/common.h. #26216 reports the cache growing past its own cap and a deterministic segfault under sustained unique-prompt vision load — the exact shape of a long CUA run — on a Blackwell 16 GB host. Two caveats before you treat it as a property of your build: it was closed as not_planned within fifteen minutes and drew no discussion, and it was reproduced on the project's server-cuda container images rather than a source build. The knobs are cheap regardless: cap the cache with -cram 2048 (or -cram 0 to disable it) and reduce -ctxcp if a many-round run climbs.

Out of memory at load

In order of what to try: drop -c from 65536 to 32768, which frees exactly 1.000 GiB of KV and is still more than twice the harness's 15,000-token floor; then step down to Fara1.5-4B-Q8_0.gguf, which frees a further 3.66 GiB and is the configuration the 12 GB cards run.

Do not reach for the usual --cache-type-k q8_0 --cache-type-v q8_0 here — it is incompatible with the --flash-attn off this recipe needs. llama-context.cpp rejects the combination outright: "quantized V cache requires flash_attn to be enabled". Quantising the K half alone is legal and needs no flash attention: block_q8_0 is one ggml_half scale plus 32 int8_t quants, 34 bytes per 32 values against f16's 64, so the K term drops from 16 to 8.5 KiB/token and the total from 32 to 24.5 KiB/token — 1.531 GiB instead of 2.000 GiB at 65,536 context. Keep the f16 projector regardless: at 0.626 GiB it is not where the savings are, and the q8_0 projector some quantisers ship trades vision fidelity in a model whose entire job is looking at pixels. If you need it out of VRAM entirely, --no-mmproj-offload keeps it on the CPU at the cost of slower image encoding — once per agent turn.

A third lever is specific to running with flash attention off: lower -ub. The F32 attention-score tensor derived in the budget section scales linearly with it, so -ub 256 cuts that upper bound from 2.000 to 1.000 GiB, and -ub 128 to 0.500 GiB, at some prompt-processing throughput. It costs nothing at decode time.

Finally, check what else holds VRAM: on a card that also drives your display, the Chromium this agent opens is competing for the same 16 GB.