How much VRAM does Fara1.5-9B need?

About 16 GB — the minimum this recipe targets.

How hard is this setup?

Advanced — follow the steps above.

Fara1.5-9B on RTX 5060 Ti 16GB: a Q8_0 Computer-Use Agent on Blackwell

What You'll Build

A local browser computer-use agent (CUA) on a single RTX 5060 Ti: llama-server hosting Fara1.5-9B as a Q8_0 GGUF plus its vision projector, with Microsoft's own fara-cli harness pointed at it, driving a Playwright-launched Chromium at 1440×900. You give it a task in English; it takes a screenshot, reasons, and emits a click, a keystroke or a URL visit, then loops.

Two things separate this page from the other 16 GB recipes for this model. First, the 9B parts company with its 4B sibling on exactly this class of card: the 4B leads unquantised bf16 here, and the 9B cannot, because 16.690 GiB of bf16 weights is over a 16 GiB card before anything else is loaded. Q8_0 is the highest tier this card can hold, and what that rung costs is paid at decode time rather than in capacity — it reads 1.39× the bytes per token that the 12 GB tier's Q5_K_M does. Second, three open llama.cpp reports touch Blackwell, vision, or this exact board, and one of them names it directly. All three are scoped in Troubleshooting rather than generalised into warnings, because the differences between what was reported and what this page installs are large enough to matter.

Hardware data: RTX 5060 Ti (16GB VRAM) · Q8_0 + f16 mmproj + 65,536 context = 11.942 GiB of accountable allocations, plus a derived 1.000 GiB attention-score buffer at the -ub 256 this recipe sets = 12.942 GiB worst case, 3.058 GiB spare · See benchmark data

⚠️ This model clicks, types and submits in a real browser, and the path documented here gives it no isolation of any kind. That the model acts is the product, not a side effect, and this is the thing to settle before anything else on this page.

The model card is blunt about the consequence: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." That sentence closes its Recommended Deployment: MagenticLite section, which lists sandboxing, allow-lists, watch-mode and a pause control as MagenticLite features and then says: "If you integrate Fara1.5-9B directly, you're responsible for these controls." The Responsible AI section asks for the same thing in more detail — "run Fara in an isolated container with no access to host files, environment variables, or sensitive credentials".

fara-cli is a direct integration. Its install step is playwright install, and it drives a plain local Playwright Chromium — a fresh browser profile, not your logged-in one, but no container, no allow-list and no pause control. The harness README offers the containerised option as an alternative to it, not as part of it: "instead of fara-cli you can use Magentic-UI", described there as "a sandboxed browser environment with auditable action logging and user prompts at critical points". For the CLI path the vendor's position is a recommendation, not something shipped: "We recommend running it in a sandboxed environment, monitoring its execution, and avoiding sensitive data or high-risk domains."

So on the fara-cli path you are the boundary. Practically: a throwaway OS user or a VM, signed out of everything, low-stakes public sites, and eyes on the run rather than leaving it unattended. The command in Running passes --headful for exactly that last reason.

ℹ️ Not a general-purpose vision chatbot. Fara1.5-9B is fine-tuned for one loop: screenshot in, <tool_call> out. It is in our multimodal vertical because it takes images and text and emits text, but it is trained against a specific system prompt and a fixed computer_use action schema. Per the model card: "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree." A Fara install that only serves text is a broken install — the --mmproj step below is mandatory, and Step 5 exists to prove an image really reached the model before you let it drive anything.

⚠️ Runtime status — read before you commit an evening to this. llama.cpp support for this architecture is verifiable statically, and this recipe shows that evidence rather than asserting it. What does not exist anywhere public is a report of Fara1.5 decoding an image under llama-server or llama-mtmd-cli on any GPU. What does exist is llama.cpp #25717 — open, community-filed, one comment — reporting llama-server crashing with a CUDA illegal memory access on vision requests for a larger model of the same architecture family with an mmproj. Its Hardware line reads "RTX 5070 Ti 16 GB (Blackwell, sm_120), driver 610.74": a different board at the same compute capability and the same VRAM tier as this one. It is not about Fara1.5, and Troubleshooting states exactly what transfers, what does not, and why the thing that helps most is a property of Fara's own harness rather than anything you type. Separately, #26257 does name this exact board — but only in a dual-GPU machine, and this recipe is single-GPU; see Troubleshooting. Treat your first run as the test, and see Results for why there is no speed figure here either.

Requirements

Component	Minimum	This recipe
GPU	16GB VRAM, CUDA compute 12.0 (sm_120)	RTX 5060 Ti 16GB — not measured; budget below is derived (/contribute)
RAM	16GB (weights are mmap'd on their way to VRAM; Playwright's Chromium sits on top), 32GB comfortable	—
Storage	10.46 GB (Q8_0 + f16 mmproj)	10,464,149,152 B = 9.745 GiB, plus the harness and Chromium
Software	CUDA Toolkit 12.8+, llama.cpp b10087+ (CUDA), Python 3.10+, Playwright	—

Microsoft ships safetensors only, and its own hardware line for this size asks for "A GPU with enough memory for a 9B model in bf16 (A6000, A100, H100, and B200 have been tested)". That is not a request a 16 GB card can meet for the 9B — the arithmetic below says so explicitly, and by a margin no configuration setting can close — so this recipe takes the highest-precision GGUF that does fit and states what that costs.

On the RAM row. The weights are mmap'd, so most of that 10.46 GB is file-backed rather than resident. The reason to want 32 GB is llama-server's prompt cache: its default ceiling is 8 GiB of system RAM (cache_ram_mib = 8192 in common/common.h), and a CUA sending a distinct screenshot every step is precisely the workload that fills it — which is what #26216 was about. On a 16 GB machine cap it with --cache-ram; see Troubleshooting.

Why the 9B on a 16 GB card

The 9B is the middle of three sizes and has to earn the tier. The model card's own evaluation table is the cleanest statement of what it buys:

Model	WebVoyager	Online-Mind2Web	WebTailBench
Fara1.5-4B	80.8	57.3	27.4
Fara1.5-9B	86.6	63.4	32.3
Fara1.5-27B	89.3	72.3	40.2

The 4B→9B step is the larger of the two on WebVoyager (+5.8 against +2.7) and comparable on the other two, so the 9B is where most of the family's headroom is bought — and it is the biggest size this card can serve. The 27B is out of reach here by arithmetic, not by preference: its Q4_K_M build is 16.329 GiB before a 0.864 GiB projector, over the card outright. Full methodology is in the Fara1.5 paper.

What the tier costs on this card is unusually easy to state, because the 4B and 9B are architecturally identical everywhere except width. Both declare num_key_value_heads: 4, head_dim: 256, 32 blocks and full_attention_interval: 4 in config.json — so the per-token KV cost of the 9B is exactly the same as the 4B's, and so is its fixed recurrent state. The entire difference is weights. You pay it once, in VRAM you would otherwise leave idle, and nothing at all in running cost per token of context.

How it fits in 16 GB — the arithmetic

There is no benchmark for this pair yet — /check reports unknown with zero benchmark rows — so every number below is arithmetic from the artifacts' own metadata and from llama.cpp's source, not a measurement. Reproduce it rather than trusting it. All totals are GiB (2³⁰ bytes), the unit a "16GB" NVIDIA card is really in, so they compare directly against 16.

Weights. Byte counts read from the bartowski GGUF tree on the day this was written:

File	Bytes	GiB
`Fara1.5-9B-Q4_K_M.gguf`	5,910,783,104	5.505
`Fara1.5-9B-Q5_K_M.gguf`	6,852,928,640	6.382
`Fara1.5-9B-Q6_K.gguf`	7,700,259,968	7.171
`Fara1.5-9B-Q8_0.gguf`	9,545,983,104	8.890
`Fara1.5-9B-bf16.gguf`	17,920,697,216	16.690
`mmproj-Fara1.5-9B-f16.gguf`	918,166,048	0.855

(HuggingFace displays these as decimal GB — 9.55 GB for the Q8_0 file — which is the same bytes, not a different number.)

KV cache — smaller than you expect, because this is a hybrid model. config.json carries a layer_types array of 24 linear_attention + 8 full_attention entries, and llama.cpp agrees: LLM_ARCH_QWEN35 maps to "qwen35" in src/llama-arch.cpp, and src/models/qwen35.cpp assigns layers with hparams.is_recr_impl[i] = (i < hparams.n_layer()) && ((i + 1) % full_attn_interval != 0). With full_attention_interval = 4 and 32 blocks that leaves layers 3, 7, … 31 — 8 attention layers. Only those carry a cache that grows with context:

K+V per token per attention layer = 2 × n_head_kv(4) × head_dim(256) × 2 B (f16) = 4,096 B
× 8 full-attention layers                                                        = 32 KiB / token

n_ctx =  32,768 → 1.000 GiB
n_ctx =  65,536 → 2.000 GiB
n_ctx = 131,072 → 4.000 GiB
n_ctx = 262,144 → 8.000 GiB

A non-hybrid 32-layer model with the same head geometry would cost four times that. Context on this model is cheap — every doubling costs exactly 1.000 GiB more than the last.

Recurrent state — fixed, and four times bigger than a single sequence's worth. The other 24 layers hold a context-independent state, sized by n_embd_r() / n_embd_s() in src/llama-hparams.cpp from this file's own GGUF keys (ssm.conv_kernel 4, ssm.inner_size 4096, ssm.group_count 16, ssm.state_size 128 — matching linear_conv_kernel_dim, linear_value_head_dim × linear_num_value_heads, linear_num_key_heads and linear_key_head_dim in the source config):

conv state : (4−1) × (4096 + 2×16×128)  =  24,576 elems
ssm  state :         128 × 4096         = 524,288 elems
per layer  : 548,864 × 4 B (F32)        = 2.094 MiB
× 24 recurrent layers                   = 50.25 MiB   ← per sequence
× n_seq_max = 4                         = 201.00 MiB = 0.196 GiB

That ×4 is not optional and it is easy to miss. src/llama-model.cpp sizes the recurrent cache with /* recurrent_rs_size */ std::max((uint32_t) 1, cparams.n_seq_max), and llama-server does not leave n_seq_max at 1: common/arg.cpp sets params.n_parallel = -1 for the server example, and tools/server/server.cpp then resolves that with if (params.n_parallel < 0) { params.n_parallel = 4; params.kv_unified = true; }. So the recurrent state is allocated once per slot — four times — while the KV cache is not multiplied, because the same default makes it unified: a unified cache holds -c tokens in total rather than per sequence. That line is also why -c 65536 gives a single trajectory the whole 65,536 tokens instead of a quarter of them, and it is why Step 3 passes no --parallel.

The whole trade, in one table. Accountable allocations (text weights + f16 projector + KV + the 0.196 GiB recurrent state) in GiB, against a 16 GiB card. Each cell is the sum of the exact byte counts above, rounded once at the end:

Context	Q4_K_M	Q5_K_M	Q6_K	Q8_0	bf16
16,384	7.056	7.934	8.723	10.442	18.241
32,768	7.556	8.434	9.223	10.942	18.741
65,536	8.556	9.434	10.223	11.942	19.741
131,072	10.556	11.434	12.223	13.942	21.741
262,144 (native)	14.556	15.434	16.223	17.942	25.741

Read that as three facts rather than twenty numbers.

bf16 is excluded by arithmetic, not by margin. The unquantised build is 16.690 GiB of weights alone — 0.690 GiB over a 16 GiB card before a projector, a single KV byte or a compute buffer. There is no context setting that rescues it, because context is not what breaks it. This is the one place the 9B behaves differently from its 4B sibling on the identical card: the 4B's bf16 file is 7.846 GiB and its recipe for this card leads with it, partly because an unquantised model never enters the quantised matmul path that #23385 is about. At 9B neither the fit nor that side-benefit is available, and this recipe says so rather than implying the tiers are equivalent. See Troubleshooting.
Q8_0 is therefore the top rung, and it fits with room. 11.942 GiB at 65,536 context. Against the published 12 GB recipe for this model — Q5_K_M at 32,768 — 16 GB buys two quant rungs and a doubling of the window. How much room is left over is a Blackwell-specific question, and the next section answers it.
The native 262,144-token window stays out of reach, and buying it would cost more than it is worth. Q6_K reaches 16.223 GiB there (over the card); Q5_K_M lands at 15.434 and Q4_K_M at 14.556, leaving 0.566 and 1.444 GiB respectively — not enough working room to load. So the native window costs three quant rungs and still does not leave headroom. Declined.

Why quant before context, explicitly. This model's output is pixel coordinates: the card describes it predicting "the next action with grounded arguments (e.g., pixel coordinates for a click)", and its training data includes "Curated datasets for predicting actions and pixel coordinates from screenshots". No public evaluation measures what any quantisation tier costs a grounding model, so every rung down the ladder is an unmeasured risk. Context, by contrast, is a measured need with a documented floor: the harness README asks for "at least 15000 tokens", the card says "We only keep the most recent 3 screenshots" in history, and the per-round text is bounded. 65,536 is 4.3× the stated floor with a hundred rounds of slack.

The Blackwell tax: what `--flash-attn off` costs, and what it does not buy

Step 3 disables flash attention. That decision is argued in Troubleshooting; this section is only about what it does to the budget, because on this card it is the difference between comfortable and tight, and it is the term that decides this recipe's -ub.

Flash attention is on unless you say otherwise. The flag's documented default is auto, and src/llama-context.cpp sets cparams.flash_attn = params.flash_attn_type != LLAMA_FLASH_ATTN_TYPE_DISABLED, so both AUTO and ENABLED switch it on. Turning it off routes attention through the non-flash branch of build_attn_mha in src/llama-graph.cpp, which materialises the attention-score tensor and pins it to F32 — ggml_mul_mat_set_prec(kq, GGML_PREC_F32), with the source comment "this op tends to require high floating point range". That tensor is n_kv × n_tokens × n_head × 4 B, the graph is reserved for the worst case, and qwen35.attention.head_count = 16, so with n_tokens at the physical batch size:

context	at `-ub 512` (default)	at `-ub 256`	at `-ub 128`
32,768	1.000 GiB	0.500 GiB	0.250 GiB
65,536	2.000 GiB	1.000 GiB	0.500 GiB
131,072	4.000 GiB	2.000 GiB	1.000 GiB

It is linear in context, and that is what makes it dangerous rather than merely annoying. The KV bytes themselves do not change — attn_v_trans is set from !cparams.flash_attn in src/llama-model.cpp, which transposes V rather than resizing it — but this buffer doubles every time you double the window, on top of a KV cache that is already doubling.

Apply it to the lead:

configuration	accountable	+ kq bound	total	left on a 16 GiB card
Q8_0 @ 65,536, `-ub 512`	11.942	2.000	13.942	2.058
Q8_0 @ 65,536, `-ub 256`	11.942	1.000	12.942	3.058
Q8_0 @ 131,072, `-ub 256`	13.942	2.000	15.942	0.058 ✗
Q6_K @ 65,536, `-ub 256`	10.223	1.000	11.223	4.777

Three conclusions come out of that table.

-ub 256 is the cheap half of the tax, so this recipe pays it up front. Halving the physical batch halves this buffer and costs nothing but prompt-processing throughput — it does not touch the context window, the quant tier, or the trajectory length. Dropping from Q8_0 to Q6_K would free more (1.719 GiB against 1.000), but it spends the one thing this page argued is worth protecting.
131,072 is dead on this card, and it was alive on the Ada one. Its accountable total of 13.942 GiB fits a 16 GiB card on paper; add the buffer and it does not. This is the concrete cost of the flag, and it is why a reader comparing this page against a same-capacity Ada recipe will find a shorter window recommended here.
The remaining 3.058 GiB is not free space you can plan against to the last byte. Treat the kq figure as an upper bound on one tensor, not a measurement: the graph allocator may hold more than one buffer of that shape, and at decode time n_tokens is 1 so the term nearly vanishes. On the other side of the ledger, the CUDA context, the vision encoder's working set and — if this card also drives your display — a GPU-composited Chromium all come out of that 3.058 GiB and none of them is derived here. Read the CUDA0 compute buffer size lines llama-server prints on startup for the real figures on your build, and report what you see.

One thing -ub 256 is explicitly not: a crash mitigation. #25717's only long-running survivor used larger power-of-two batches, not smaller ones — its author notes that configuration "survives full production runs (20 sequential vision requests with json_schema grammar, ~800-token instruction after the image)". Lower -ub here buys memory. Nothing else.

How the context is actually spent

A CUA sends a screenshot every step, so the per-image cost decides whether the budget above is realistic. The harness fixes the browser viewport at 1440×900 (viewport_width: int = 1440, viewport_height: int = 900 in fara15_agent.py), matching the card's note that "The screen resolution Fara is most commonly trained with is" 1440×900. It pre-resizes with Qwen's smart_resize at PATCH_SIZE = 16 and MERGE_SIZE = 2, so the factor is 32 and 900 becomes 896. llama.cpp then counts merged patches per axis in tools/mtmd/clip.cpp, where the PROJECTOR_TYPE_QWEN3VL branch returns (img->nx() / params.patch_size) / 2 across and (img->ny() / params.patch_size) / 2 down, and clamps the result with hparams.set_limit_image_tokens(8, 4096):

across : (1440 / 16) / 2 = 90 / 2 = 45
down   : ( 896 / 16) / 2 = 56 / 2 = 28
tokens : 45 × 28                 = 1,260 per screenshot
× max_n_images = 3               = 3,780 tokens resident

The harness's own conservative default is image_token_estimate: int = 1500, a useful sanity check on that arithmetic, and the card confirms the retention: "We only keep the most recent 3 screenshots".

So images are capped. What grows is the text trajectory: every round appends the model's reasoning, its tool call and an observation (max_observation_chars: int = 1000), and fara-cli defaults to --max_rounds 100. That is what 65,536 is sized for.

One consequence matters for the crash risk on this card and is easy to miss: because the image history is capped but the text history is not, the number of text tokens sitting in front of each screenshot grows with every round. That is one of the three variables #25717 identifies as a trigger. Step 5's second test probes it deliberately.

Installation

1. Build llama.cpp with CUDA 12.8+ for sm_120

This is the one step that genuinely differs on Blackwell. The RTX 5060 Ti is compute capability 12.0 — NVIDIA's CUDA GPUs table lists it in the same 12.0 group as the rest of the GeForce RTX 50 series — and llama.cpp's CUDA CMakeLists annotates that target as "Blackwell, needs CUDA v12.8, FP4 tensor cores". Check the toolkit first, because this is the only reliable way to check it:

nvcc --version    # must report release 12.8 or newer

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

Two details worth knowing rather than pasting:

Pass 120 explicitly — but not because the default would silently miss your card. It would not. GGML_NATIVE defaults ON in ggml/CMakeLists.txt unless you are cross-compiling or have SOURCE_DATE_EPOCH set, and the CUDA CMakeLists then takes set(CMAKE_CUDA_ARCHITECTURES "native") before it ever reaches the toolkit-version list — so the default path targets whichever card is attached at compile time, and an inadequate toolkit fails the build loudly. What naming 120 buys is a build that does not depend on what was plugged in when you compiled, which matters in a container or on a separate build host, and it closes the one soft failure the native path has: llama.cpp's build guide notes that when nvcc cannot detect the GPU you get only a warning — "Cannot find valid GPU for '-arch=native', default arch is used" — and then a binary built for something else.
You write 120, the build system compiles 120a. The same file rewrites any plain 12X architecture into its architecture-specific 12Xa form because, in its own words, "the Blackwell FP4 tensor core instructions are not forwards compatible". You do not type the a, and you should not be surprised to see 120a-real in the CMake status line. Do not read that line as a toolkit check — Troubleshooting explains why it proves nothing.

Both halves of the model are first-class in mainline — worth confirming before you download 10 GB:

grep -n 'LLM_ARCH_QWEN35,' src/llama-arch.cpp        # -> { LLM_ARCH_QWEN35, "qwen35" }
grep -n 'qwen3vl_merger' tools/mtmd/clip-impl.h      # -> PROJECTOR_TYPE_QWEN3VL

Build from a release at or after b10087 — the release this GGUF was quantised on, per the quantiser's card. That is also newer than the b10002 / b10016 builds #25717 was filed against, which is one reason to test rather than assume the report reproduces.

A note on FP4, since Blackwell is why people ask. The FP4 tensor cores in this silicon are real and llama.cpp does use them: GGML_TYPE_NVFP4 is a shipped quant type in ggml/include/ggml.h, and mmq.cu gates a native path on your architecture with use_native_fp4 = blackwell_mma_available(cc) && (src0->type == GGML_TYPE_MXFP4 || src0->type == GGML_TYPE_NVFP4). What is missing is not the loader — it is an artifact: no NVFP4 build of Fara1.5-9B exists to load. And FP4 is a 4-bit tier, i.e. the opposite direction from what this card's 16 GB is for. Build for 120 because that is your architecture, not because it unlocks a 4-bit path.

2. Download the weights and the projector

Both files are required. The projector is a separate artifact — downloading only the text GGUF gives you a model that cannot see.

pip install -U huggingface_hub

hf download bartowski/Fara1.5-9B-GGUF \
  Fara1.5-9B-Q8_0.gguf mmproj-Fara1.5-9B-f16.gguf \
  --local-dir ~/models/fara1.5-9b

Microsoft publishes no first-party GGUF for Fara1.5 — the microsoft org ships Fara1.5-4B, Fara1.5-9B and Fara1.5-27B as safetensors only. Several independent community quantisers have converted the 9B including its vision tower, each publishing a matching mmproj; independent conversions succeeding is the strongest available evidence that the vision path converts cleanly, and it is still not a runtime test.

You can confirm the artifact you just downloaded declares the projector llama.cpp implements:

strings ~/models/fara1.5-9b/mmproj-Fara1.5-9B-f16.gguf | grep -m1 qwen3vl_merger

The same repo also ships mmproj-Fara1.5-9B-bf16.gguf at 921,704,992 B — 3 MiB larger. At the same two bytes per value, IEEE binary16 spends three more bits on the mantissa than bfloat16 does and three fewer on the exponent: a finer grid over a narrower range. This recipe takes the f16 build, as every other recipe for this model does; substitute the bf16 one and add 3 MiB to every figure above.

3. Serve it

./build/bin/llama-server \
  -m ~/models/fara1.5-9b/Fara1.5-9B-Q8_0.gguf \
  --mmproj ~/models/fara1.5-9b/mmproj-Fara1.5-9B-f16.gguf \
  -ngl 99 -c 65536 --flash-attn off -ub 256 \
  --host 127.0.0.1 --port 5000 \
  --image-min-tokens 1024 \
  --reasoning-format none \
  --temp 0

Five of those are load-bearing and none of them is a default:

--flash-attn off — the flag that is specific to this silicon. Flash attention on is the only variable in #25717's matrix that crashed on its own, with a short prompt and no other trigger present. Turning it off removes that unconditional failure and nothing more; it is not protection, and Troubleshooting is explicit about the two rows that crash with it off. Its memory cost is derived above.
-ub 256 — halves the F32 attention-score buffer that --flash-attn off brings with it, from 2.000 GiB to 1.000 GiB at this context. Prompt-processing throughput is the only thing it costs. See the table above.
No --parallel, and no --kv-unified either. A unified KV cache is what you want — it is the layout of both configurations in #25717's matrix that completed, and it keeps the whole window available to one request. You get it by saying nothing: the server's auto-block sets n_parallel = 4 and kv_unified = true together, as quoted above. Passing --kv-unified is a no-op. Passing --parallel N is worse than a no-op: it suppresses that auto-block, so kv_unified falls back to false, n_ctx_seq becomes n_ctx / n_seq_max, and --parallel 4 leaves 16,384 tokens per request — barely over the harness's 15,000 floor, with --parallel 8 under it. --parallel 1 is also present in both of the flash-attention-off rows that crashed.
--reasoning-format none — mandatory. Without it the agent loop silently receives nothing; see Troubleshooting for the mechanism. The server documents this value as "none: leaves thoughts unparsed in message.content".
--image-min-tokens 1024 — this projector ships no minimum-pixel key, so llama.cpp falls back to the floor set by hparams.set_limit_image_tokens(8, 4096) in the PROJECTOR_TYPE_QWEN3VL branch of tools/mtmd/clip.cpp, and warns four lines later in the same branch: "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks". Fara is a grounding model, so that warning is aimed precisely at this case. At the harness's viewport the image is already 1,260 tokens and the floor never binds; it protects you if you shrink the viewport.

And one that is not on the line: -c 65536, not the model's native 262,144. At Q8_0 the native window is 17.942 GiB accountable and will not load. See the budget table.

If you would rather not manage files, -hf bartowski/Fara1.5-9B-GGUF:Q8_0 fetches the model, and per the server docs "mmproj is also downloaded automatically if available".

4. Confirm what actually loaded

./build/bin/llama-server --version    # build number >= 10087

Three lines of the startup log are worth reading before going further: the CUDA0 device line (it should name your 5060 Ti — if the model loaded on CPU, go back to Step 1), the flash_attn line (it must say disabled; if it does not, your flag did not take), and the mmproj load stage.

5. Prove the vision path works — twice, and the second time is the one that matters

Check that the server advertises multimodal support, then send a real image. The server docs make the first part a client-side requirement rather than an optional nicety: "Clients should check /models or /v1/models for the multimodal capability before a multimodal request." Run GET /props alongside it — it reports which modalities loaded rather than merely that some did, which is the more useful signal when a projector silently failed to load.

curl -s http://127.0.0.1:5000/v1/models | grep -o multimodal

# supplementary — names the modality that actually loaded
curl -s http://127.0.0.1:5000/props | python3 -c "import sys,json; print(json.load(sys.stdin)['modalities'])"
# expect: {'vision': True}

# test A — short prompt, image first (the ordering Fara's own harness uses)
curl -s http://127.0.0.1:5000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "fara",
  "temperature": 0,
  "messages": [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}},
    {"type": "text", "text": "Describe this image in one sentence."}
  ]}]
}' | python3 -m json.tool

Test A is not sufficient on this card. In #25717's matrix a short prompt with flash attention off completed, and the same configuration with roughly 800 text tokens in front of the image did not. Test B is deliberately harsher than what the harness sends — Fara puts the screenshot before the text, which is the ordering that survived in the report, whereas test B puts the text first. Run it anyway: it is the cheapest way to find out whether that failure exists on your build at all.

# test B — ~800 tokens of text BEFORE the image, the reported crashing order
python3 - <<'PY'
import json, urllib.request
long_text = "Describe precisely. " * 200
body = {"model": "fara", "temperature": 0, "max_tokens": 200,
        "messages": [{"role": "user", "content": [
            {"type": "text", "text": long_text},
            {"type": "image_url", "image_url": {"url":
             "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}]}]}
req = urllib.request.Request("http://127.0.0.1:5000/v1/chat/completions",
        data=json.dumps(body).encode(), headers={"Content-Type": "application/json"})
print(urllib.request.urlopen(req, timeout=300).read()[:400])
PY

If both replies describe the picture, you have cleared the risk this page is most worried about. If test B kills the server — watch the log for a find_slot: non-consecutive token position line immediately before the CUDA error — go to Troubleshooting. That would be the reported failure reproduced on a different architecture and a newer build, which is worth a comment on the upstream issue.

6. Install the harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Skip the README's [vllm] extra — that path serves BF16 safetensors on datacenter cards, which is exactly what this recipe cannot do on 16 GB.

Running

Two clients share the same llama-server endpoint. Path A (fara-cli) is the smoke test, with no isolation. Path B (Magentic-UI) is what the vendor recommends for anything real. They are not equivalent in risk; pick deliberately.

Path A — `fara-cli`, for a first smoke test

source .venv/bin/activate
fara-cli \
  --base_url http://127.0.0.1:5000/v1 \
  --api_key sk-no-key-required \
  --model fara \
  --task "find the current UTC time on time.is and tell me" \
  --output_folder ./runs/first \
  --save_screenshots \
  --headful \
  --max_rounds 15

--headful is in that command on purpose. The harness runs the browser headless by default — run_fara.py declares the flag with the help text "Run the browser in headful mode (show GUI, default is headless)", and both launch sites read headless=not args.headful; the README likewise shows it as an optional [--headful]. Either way this drives a real, unsandboxed Chromium on your machine — the danger is identical with or without the flag. What the flag changes is whether you can see it, and the safety callout above asks you to keep eyes on the run. Drop it once you trust the loop and are reading the trajectory files instead.

Each step writes a screenshot and a data_point.json into --output_folder, which is how you tell a grounding failure (clicks landing in the wrong place) from a reasoning failure. Start with a low --max_rounds for a second reason: if the prompt-cache growth in #26216 is real on a source build, it shows up on long runs, not short ones.

Keep the task on a low-stakes public site and stay signed out. Fara is trained to stop and ask before entering personal information, making payments, submitting forms, signing in or sending messages; if you see it sail through one of those, stop the run — that is a wiring problem in your prompt path, not a quirk to tolerate.

Path B — Magentic-UI, for the container

Everything the model card assumes — the containerised browser, domain allow-lists, watch-mode and a pause control — lives in Magentic-UI, not in fara-cli. That is also where a Docker requirement would come from; there is no Docker step in Path A because Path A genuinely does not use one.

Magentic-UI takes the same OpenAI-compatible endpoint you started above, so nothing on this page changes except the client. Its setup is documented in its own repository rather than reproduced here, because this recipe has not verified those steps end to end on this GPU — give it http://127.0.0.1:5000/v1 as the model endpoint. Whichever client you use, match the trained resolution: the card is explicit — "Match this in your sandbox for the most reliable grounding."

Results

Speed: omitted. No published measurement of Fara1.5-9B exists for this card or for any consumer GPU, and Microsoft publishes no throughput figures at all — the card names A6000, A100, H100 and B200 only as hardware its bf16 serving path was tested on, which says nothing about a Q8_0 build on sm_120. Note also that --flash-attn off is not free in time as well as memory, and nobody has measured that either. If you run this, please contribute your numbers so this line can be replaced with a measurement.
VRAM usage: 9.745 GiB of weights and projector; 11.942 GiB accountable including KV and the 4-slot recurrent state at 65,536 context; 12.942 GiB once the derived 1.000 GiB upper bound on the flash-attention-off attention-score tensor at -ub 256 is included, leaving 3.058 GiB of the card. All of that is derivation, not an observed peak — the CUDA context, the vision encode graph and anything else on the card sit on top. Track the empirical picture at /check/fara1-5-9b/rtx-5060-ti.
Quality notes: the vendor's published scores (86.6 WebVoyager, 63.4 Online-Mind2Web, 32.3 WebTailBench) are for the bf16 model on datacenter hardware. Neither the hardware nor the precision transfers here, and no public evaluation measures what Q8_0 costs a model whose job is predicting exact pixel coordinates — which is why this recipe leads the highest tier that fits rather than the cheapest one. The vision path is structurally proven, not runtime-proven: mainline llama.cpp registers the architecture and the qwen3vl_merger projector, several quantisers converted it independently, and llama-server documents the image_url content parts the harness sends — but nobody has published a report of Fara1.5 grounding a click under llama-server.
Language: English only — the card puts languages other than English out of scope.

For the full benchmark data, see /check/fara1-5-9b/rtx-5060-ti.

Troubleshooting

The server dies on an image request with `CUDA error: an illegal memory access was encountered`

This is why Step 5 has two tests. llama.cpp #25717 reports llama-server crashing with exactly that error on vision requests, preceded in the log by find_slot: non-consecutive token position N after N-1. The reporter bisected seven configurations; in his own summary the crash depends on "flash attention", "KV cache layout (unified vs split)" and — the surprising one — the amount of text tokens placed before the image in the prompt. Two rows completed: flash attention off with the default unified cache and a short text prefix, and one where the same long prompt was sent with the image as the first content part.

Read it with its limits stated, in both directions.

The hardware is a 16 GB Blackwell card, but not this one. The report's own Hardware line is "RTX 5070 Ti 16 GB (Blackwell, sm_120), driver 610.74" — the same compute capability and the same VRAM tier as this board, a different model of card. That is enough to take the report seriously and not enough to treat it as a prediction about your machine.
It is a community report with no maintainer engagement. One comment, and that comment is the reporter adding his own reproducer.
It is a different llama.cpp architecture, not merely a different size. The reported model is unsloth/Qwen3.5-35B-A3B-GGUF, a sparse MoE that registers as qwen35moe — a separate arch enum from Fara's qwen35 — with 40 blocks against Fara's 32 and 2 KV heads against Fara's 4. Its projector is a different and larger file.
The configuration is not this one either. Every row of that matrix ran --n-cpu-moe 24 --batch-size 4096 --ubatch-size 4096 on Windows builds b10002/b10016; this recipe is fully offloaded on b10087 or newer at -ub 256, a sixteenth of the physical batch. The reporter's own "Related" line points at #19929, a closed batch-size-dependent M-RoPE issue whose partial fix he notes was already in his builds. Batch size is a live variable across this family of reports and his was unusually large.
The mechanism, however, transfers better than any of that suggests. The non-consecutive token position line that precedes every crash is emitted by src/llama-memory-recurrent.cpp — the recurrent half of the hybrid cache, not the attention half, and nothing to do with experts. Fara routes 24 of its 32 layers through exactly that memory. And the M-RoPE positioning the reporter blames is present here too: the source config sets mrope_interleaved: true and the GGUF carries a four-entry rope.dimension_sections. So the right posture is neither "this will happen to you" nor "different model, ignore it" — it is "both ingredients are present, nobody has run the combination, so test it in Step 5".

What helps most is the message ordering, not the flag. Row 7 is the only long-prompt survivor in that matrix, and what it did differently was put the image first. Fara's harness does exactly that, at both of its call sites — content=[ImageObj.from_pil(scaled_screenshot), task_instruction] for the opening turn and content=[ImageObj.from_pil(scaled_screenshot), text_prompt] for every round after it, in fara15_agent.py. So the client you are pointing at this server produces the surviving configuration by construction, and you did not have to do anything to get it. The residual doubt is that Fara's system prompt is long text and still precedes the image in the flattened token sequence, and that prefix grows every round — so the shape is close to row 7 rather than identical to it.

--flash-attn off is still in Step 3, but understand what it buys: it removes the condition that failed unconditionally (rows 1–3 are every flash-attention-on row, and all three crash), and nothing more. It is not protection — rows 5 and 6 crash with it off, at 774 tokens of text before the image, and what those two rows have in common is --parallel 1, which is why Step 3 passes no --parallel at all. If you send Fara-shaped prompts through some other client that puts text first, do not read the flag as a guarantee.

A fatal abort in the MMQ kernels, or a CUDA invalid-value at kernel launch

llama.cpp #23385 is open and names Blackwell in its title — "Fatal MMQ crashes on Blackwell (RTX 5090/5080) due to unhandled sharedMemPerBlockOptin driver bug". Early Blackwell drivers return a corrupted cudaDeviceProp.sharedMemPerBlockOptin — either 0 or an overflowed value — and llama.cpp maps that straight onto the shared-memory budget its quantised matmul kernels dispatch against. ggml-cuda.cu still assigns info.devices[id].smpbo = prop.sharedMemPerBlockOptin unguarded, though mmq.cu has since added a floor ("MMQ tiles require at least 48 KiB per-block shared memory; fall back to BLAS otherwise.") that turns the 0 case into a cuBLAS fallback rather than an abort.

This is the one place where the 9B is more exposed than the 4B on the same silicon, and it is worth being explicit about. The 4B's recipes for 16 GB Blackwell cards lead unquantised bf16, which never enters that code path at all: ggml_cuda_should_use_mmq in mmq.cu switches on the weight type and returns false for anything that is not a quantised format. GGML_TYPE_Q8_0 is on its accepted list. Because bf16 is over this card by arithmetic at 9B, that escape hatch does not exist here — the lead quant does reach MMQ, so the report is relevant to this configuration in a way it is not to the 4B's.

Two ways out, in order: update the graphics driver (the root cause is the driver, not llama.cpp); or rebuild with -DGGML_CUDA_FORCE_CUBLAS=ON, a real option in ggml/CMakeLists.txt described as "always use cuBLAS instead of mmq kernels", which short-circuits ggml_cuda_should_use_mmq to false before it inspects anything. Expect to give up some throughput for it. Note the report names other cards than this one — treat it as a driver property of the architecture to rule out, not a prediction about your board.

The build succeeds but the GPU is idle, or the server falls back to CPU

Check the toolkit first — nvcc --version must report 12.8 or newer, because Blackwell's FP4 tensor-core instructions are not forwards-compatible and an older toolkit cannot emit sm_120a code at all. But note what a successful build already tells you: with -DCMAKE_CUDA_ARCHITECTURES=120 named explicitly, a toolkit that cannot target sm_120 fails the compile rather than producing a quiet no-op, so if Step 1 completed then the toolkit is probably not your problem. One command rules the class out; then look at -ngl, at whether llama-server logged the CUDA device at startup, and at the prebuilt-binary trap below.

Whatever you do, do not try to confirm the toolkit from the configure log. The 12X → 12Xa rewrite in the CUDA CMakeLists is unconditional — it sits outside the if (NOT DEFINED CMAKE_CUDA_ARCHITECTURES) block that holds the toolkit-version gate, and Step 1 passes -DCMAKE_CUDA_ARCHITECTURES=120 explicitly, which skips that block entirely. So Replacing 120 … with 120a prints on every toolkit version and proves nothing. nvcc --version is the test.

Relatedly: build from source rather than reaching for a third-party "sm120 Blackwell" binary. #26205 is an open report of llama-server crashing silently before it starts listening on an sm_120 card under exactly that kind of prebuilt Windows build, with the official older-CUDA build giving 0% GPU utilisation on the same machine.

The agent does nothing — every step logs an empty response or `list index out of range`

This is the failure everyone hits, and it is not a bug in your setup. Fara opens a <think> block and never closes it — you can read that straight out of the chat template embedded in the GGUF you downloaded, whose generation prompt emits <think>\n with no closing tag unless enable_thinking is explicitly false. Under llama-server's default reasoning extraction the parser therefore treats the whole output — reasoning and the <tool_call> block together — as reasoning, returns it in reasoning_content, and leaves message.content empty. The harness parses message.content for <tool_call>, finds nothing, and the loop dies on round one.

Launch with --reasoning-format none, documented as "none: leaves thoughts unparsed in message.content". The raw output then begins with a stray unclosed <think> tag, which is cosmetic.

Reported for Fara1.5-27B by a community user on build b10107 in the model family's HF discussions and acknowledged there by Microsoft — swhitehead, whose HF account carries the org-member flag on that repo, asked for it to be filed upstream, which became microsoft/fara#82, still open.

The model answers but ignores the screenshot

Almost always the projector. Confirm --mmproj is on the command line, that the server logged an mmproj load stage, that /v1/models advertises the multimodal capability, and that Step 5's /props call reports vision: True. A text-only GGUF with no projector loads and runs perfectly happily — it simply cannot see, which in a CUA presents as confidently wrong actions rather than an error. Note the automatic projector download only happens on the -hf path; with -m you must pass --mmproj yourself.

Clicks land in the wrong place

Check the viewport first: the harness defaults to 1440×900 because that is what the model was mostly trained at, and a different window size changes both the grounding distribution and the token cost derived above. Then confirm --image-min-tokens 1024 is set, since llama.cpp's own warning ties sub-1024-token images to degraded grounding. Only after both should you suspect the weights — and this recipe already leads the highest quantisation tier that fits on the card, so the remaining rungs are all downward.

A second GPU in the box produces garbled output

Keep this single-GPU. llama.cpp #26257 is an open community report of garbled generation from a different Qwen-family model when this board is paired with an Ampere card in the same machine — its Hardware line lists an "RTX 5060 Ti 16GB" alongside an "RTX 3060 12GB" — where the reporter states that "running on the RTX 5060 Ti + system RAM does not result in garbled output". It is the one open llama.cpp report that names this exact board, which is why it is here; but the model in it is a 27B from a different family, the mechanism is unidentified, and two comments in nobody has reproduced it single-GPU. If you have a mixed-generation pair installed, pin the run with CUDA_VISIBLE_DEVICES=0 before you conclude anything about Fara. At 12.942 GiB this recipe has no reason to span two cards in the first place.

A long agent run climbs in system RAM, or the server dies after many steps

llama-server's prompt cache and context checkpoints are sized for chat, not for a stream of unique screenshots: the defaults are cache_ram_mib = 8192 and n_ctx_checkpoints = 32 per common/common.h. #26216 reports the cache growing past its own cap and a deterministic segfault under sustained unique-prompt vision load — the exact shape of a long CUA run — and its host line names a different Blackwell board. Two caveats before you treat it as a property of your build: it was closed as not_planned within fifteen minutes and drew no discussion, and it was reproduced on the project's server-cuda container images rather than a source build. The knobs are cheap regardless: cap the cache with -cram 2048 (or -cram 0 to disable it) and reduce -ctxcp if a many-round run climbs.

Out of memory at load

In order of what to try, with what each actually frees:

Change	Frees	Cost
`-ub 256` → `-ub 128`	0.500 GiB	slower prompt processing; nothing at decode
`-c 65536` → `32768`	1.500 GiB (1.000 KV + 0.500 kq)	still 2.2× the harness's documented 15,000-token floor
`--cache-type-k q8_0`	0.469 GiB	K-only quantisation; see the warning below
Q8_0 → `Fara1.5-9B-Q6_K.gguf`	1.719 GiB	one rung down an unmeasured grounding ladder
Q8_0 → `Fara1.5-9B-Q5_K_M.gguf`	2.508 GiB	the 12 GB recipe's lead quant
`--no-mmproj-offload`	0.855 GiB	slower image encoding, once per agent turn; frees VRAM but does not dodge #25717 — row 3 of that matrix ran this flag and crashed anyway

Take -ub first, then context — those are the cheapest and best-understood, and both are consequences of the flash-attention decision rather than of the model.

Do not reach for the usual --cache-type-k q8_0 --cache-type-v q8_0 pair here. With flash attention disabled, quantising the V half is a hard startup failure, not a slowdown: src/llama-context.cpp rejects the combination outright with "quantized V cache requires flash_attn to be enabled" and returns a null context. Quantising K alone is legal and needs no flash attention: block_q8_0 is one ggml_half scale plus 32 int8_t quants, 34 bytes per 32 values against f16's 64, so the K term drops from 16 to 8.5 KiB/token and the total from 32 to 24.5 KiB/token — 1.531 GiB instead of 2.000 GiB at 65,536 context.

Keep the f16 projector: at 0.855 GiB it is not where the savings are, and trading vision fidelity in a model whose entire job is looking at pixels is the worst rung on this list. Passing --parallel 1 would shrink the recurrent state from 201.00 MiB back to 50.25 MiB, but 0.147 GiB is the smallest lever on this page, it divides your context window, and it is present in both of #25717's crashing flash-attention-off rows — reach for the table instead.

Finally, check what else holds VRAM: on a card that also drives your display, the Chromium this agent opens is competing for the same 16 GB.

`rope.dimension_sections has wrong array length; expected 4, got 3`

This is a real open llama.cpp bug (PR #25334, titled "fix: load qwen35 Ollama GGUF exports"), but it belongs to Ollama-exported qwen35 blobs, not to the GGUF this recipe uses. If you see this error, you are loading a different file than the one downloaded in Step 2.

`ollama run fara` does nothing — there is no Ollama entry

There is no Ollama library entry for Fara1.5 in any form, and the registry manifest API has none either. The GGUF/LM Studio/Ollama line in the harness README refers to Fara-7B, the previous generation, reached with a --fara-7b flag — it is not about Fara1.5, and the two should never be mixed. llama.cpp built from source is the path this recipe documents.

Generation is slower than a 9B model should be

Three things to separate before you go bug-hunting. The first is expected: Q8_0 moves 1.39× the bytes per token that Q5_K_M does, and single-token decode on a model this size is bound by how many bytes leave memory per token — that is the cost you chose in exchange for staying off the unmeasured end of the quant ladder. The second is this page's own doing: --flash-attn off takes the slower attention path by construction, so a comparison against a card that leaves it on is not a like-for-like one. A/B against Fara1.5-9B-Q5_K_M.gguf with llama-bench before treating either as a defect. The third is a lead rather than a known property: llama.cpp issue #25162 is an open, community-reported regression against exactly the SSM kernels this architecture leans on, titled "Performance regression on Turing GPUs after 9e58d4d69 — Qwen35 SSM kernels affected". Read the scope carefully: the title says Turing, the reporter measured on an RTX 2080 Ti, the thread has two comments and both are from community accounts with no maintainer response, and nobody has posted numbers for a Blackwell card. It is a thing to check with llama-bench across a couple of builds, not a property of this one.

Running it against your own accounts

Don't. The model is trained to pause before sign-ins, payments and submissions, and that training is a safety net, not a guarantee — the card lists prompt injection embedded in page content among its known risks, and a vision-only agent has no way to distinguish a legitimate button from a hostile one that renders like it. Keep it in a container, on an allow-list, with no credentials in reach.