Fara1.5-9B on RTX 5090: BF16 Computer-Use Agent on Blackwell sm_120

What You'll Build

A local browser computer-use agent (CUA): Microsoft's Fara1.5-9B served by llama-server on an RTX 5090, with the vision projector loaded so the model actually sees browser screenshots, driven by Microsoft's own fara-cli harness against a plain local Playwright Chromium.

Two things make this page different from the 24 GB recipes for the same model. First, 32 GB pays for the mitigation: this is the one architecture in this sweep with an open report of llama-server crashing on vision requests specifically on sm_120, the fix for it is to turn flash attention off, and turning flash attention off costs memory that scales with your context window. On a 24 GB card that trade would bind; here it does not. Second, the weight tier this recipe leads — unquantised BF16 — happens to route around a second open Blackwell bug entirely, for a reason you can check in llama.cpp's own dispatch code rather than take on faith.

Hardware data: RTX 5090 (32GB VRAM) · BF16 weights + f16 mmproj = 17.545 GiB · 19.741 GiB accountable at a 65,536-token context, 21.741 GiB including the F32 attention scores this card's flash-attention-off configuration adds (both derived below) · See benchmark data

⚠️ This model clicks, types and submits in a real browser, and the path documented here gives it no isolation of any kind. That is the operating requirement, not boilerplate, and it is the thing to settle before anything else on this page.

The model card is blunt about the consequence: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." That sentence closes the card's Recommended Deployment: MagenticLite section, which lists four controls as MagenticLite features — the first being "the browser runs in a Docker container with no access to host files or environment variables" — and then says: "If you integrate Fara1.5-9B directly, you're responsible for these controls." The card's Responsible AI section asks you to "run Fara in an isolated container with no access to host files, environment variables, or sensitive credentials". Fara also reads whatever is on the page, so a hostile page is a prompt-injection surface — the card lists "Prompt injection" among its known model risks by name.

fara-cli is a direct integration. Its install step is playwright install, and it drives a plain local Playwright Chromium — no container, no allow-list, no pause control. The harness README offers the containerised option as an alternative to it rather than as part of it: "instead of fara-cli you can use Magentic-UI", described in the same sentence as "a sandboxed browser environment with auditable action logging and user prompts at critical points". For the CLI path the vendor's position is a recommendation, not something shipped: "We recommend running it in a sandboxed environment, monitoring its execution, and avoiding sensitive data or high-risk domains."

So on the fara-cli path you are the boundary. Practically: a throwaway OS user or a VM, Playwright's own fresh browser profile (never your daily one), signed out of everything, low-stakes public sites, and eyes on the run rather than leaving it unattended. The command in Running passes --headful for exactly that last reason. If you want the container, the allow-lists, watch-mode and the pause control, use Magentic-UI as the client instead.

ℹ️ Vision is the product, not a bonus. Per the model card, "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree." A Fara install that only serves text is a broken install — the --mmproj step below is mandatory, and Step 5 exists to prove that an image really reached the model before you let it drive anything.

⚠️ Runtime status, specific to this silicon. llama.cpp support for this architecture is verifiable statically and this recipe shows that evidence rather than asserting it. What does not exist anywhere public is a report of Fara1.5 decoding an image under llama-server on any GPU, nor a throughput measurement of this model on any consumer card. What does exist on sm_120 is llama.cpp #25717 — an open, community-filed report of a CUDA illegal memory access on vision requests with an mmproj, against a larger model of the same architecture family — and #23385, an open report of fatal aborts in the quantised matmul kernels whose title names this card. Neither is about Fara1.5, and Troubleshooting is explicit about what transfers and what does not. But on this card they are the two things most likely to end your evening, so Step 3's flags and Step 5's second test both exist because of them. Treat your first run as the test.

Requirements

Component	Minimum	This recipe
GPU	24GB VRAM, CUDA compute 8.0+ (Ampere or newer, for BF16 tensor cores)	— not measured; budget below is derived (/contribute)
RAM	16GB, 32GB comfortable — see the note below	—
Storage	18.84 GB (BF16 + f16 mmproj)	18,838,863,264 B = 17.545 GiB, plus the harness and Chromium
Software	CUDA Toolkit 12.8+, llama.cpp b10087+ (CUDA), Python 3.10+, Playwright	—

Three notes on that table.

min_vram_gb is 24 — not 32. It is the filter floor for the install documented here, not this card's capacity, and it is worth showing the arithmetic because the 4B recipe for this same card arrives at the same number by a different route. Two things push on it in opposite directions. Upward: the BF16 text weights are 16.690 GiB on their own, 0.690 GiB over a 16 GiB card before a projector or a single byte of cache, so 16 is excluded outright — by arithmetic, not by margin. Upward again: the --flash-attn off that Step 3 requires on this architecture makes llama.cpp materialise the attention scores in F32, and that tensor scales with context — 2.000 GiB at the 65,536-token window this page leads. Downward: the window this page leads is deliberately not the largest one that fits. Net, the documented configuration accounts for 21.741 GiB, which clears a 24 GiB card with 2.3 GiB to spare, so the floor is 24. (The 4B page reaches 24 the other way round: its weights fit 16 GB comfortably and it is purely the scores term, at a 131,072-token window, that lifts it over the line.)

On the RAM row. The weights are mmap'd, so most of that 18.84 GB is file-backed rather than resident in VRAM's shadow. The reason to want 32 GB of system RAM is llama-server's prompt cache: its default ceiling is 8 GiB of system RAM (cache_ram_mib = 8192 in common/common.h), and a CUA sending a distinct screenshot every step is exactly the workload that fills it. On a 16 GB machine cap it with --cache-ram; see Troubleshooting.

And the GPU cell is not a measurement — /check reports unknown with zero benchmark rows, and nothing on this page was run on hardware by us.

Microsoft ships safetensors only, all of it in BF16: the repo's own config.json declares "dtype": "bfloat16" for the text model. Its Quickstart serves those with vLLM and names datacenter hardware for the job: "A GPU with enough memory for a 9B model in bf16 (A6000, A100, H100, and B200 have been tested)", and the card's highlights claim only that "9B parameters means it runs on a single A100/H100/B200 with room for the screenshot history". Note the dtype in the vendor's own reference invocation, vllm serve microsoft/Fara1.5-9B --dtype bfloat16: BF16 is what Microsoft runs, and it is what this card can afford without thinking about it.

What 32 GB actually buys

There is no benchmark for this pair, so every number below is arithmetic from the artifacts' own metadata and llama.cpp's source, not a measurement. Reproduce it rather than trusting it. All totals are GiB (2^30 bytes), the unit a "32GB" NVIDIA card is really in, so they compare directly against 32.

Weights. Byte counts from the bartowski GGUF tree:

Component	Bytes	GiB
Fara1.5-9B-bf16.gguf	17,920,697,216	16.690
mmproj-Fara1.5-9B-f16.gguf	918,166,048	0.855
Weights total	18,838,863,264	17.545

KV cache — and this is where Fara1.5 is unusual. Its qwen35 architecture is a hybrid: llm_arch_is_hybrid() in src/llama-arch.cpp returns true for LLM_ARCH_QWEN35, and src/models/qwen35.cpp marks layers recurrent off the full_attention_interval. The source config.json spells the split out layer by layer: its layer_types array holds 24 linear_attention entries and only 8 full_attention, against num_hidden_layers: 32 and full_attention_interval: 4. Only those 8 carry a cache that grows with context.

With num_key_value_heads = 4 and head_dim = 256:

K+V per token per attention layer = 2 x 4 x 256 x 2 B (f16) = 4,096 B
x 8 full-attention layers                                   = 32 KiB / token

A non-hybrid 32-layer model with the same head geometry would cost four times that. Context on this model is cheap — doubling -c costs exactly 1.000 GiB per doubling from 32,768 — and it is cheap at 9B for precisely the same reason it is cheap at 4B: this family keeps hidden_size 4096, head_dim 256 and the same 24/8 layer split across sizes, and grows intermediate_size instead. The per-token cache of the 9B is byte-for-byte the 4B's.

Recurrent state — fixed, and allocated four times over. The other 24 layers hold a context-independent state, sized by n_embd_r() / n_embd_s() in src/llama-hparams.cpp from the artifact's own GGUF keys (ssm.conv_kernel 4, ssm.inner_size 4096, ssm.group_count 16, ssm.state_size 128 — the config's linear_conv_kernel_dim, linear_num_value_heads x linear_value_head_dim, linear_num_key_heads and linear_key_head_dim respectively). qwen35 sets no n_embd_head_kda, so both fall through to the Mamba forms:

conv state : (4-1) x (4096 + 2 x 16 x 128) =  24,576 elems
ssm  state :         128 x 4096            = 524,288 elems
per layer  : 548,864 x 4 B (F32)           = 2.094 MiB
x 24 linear-attention layers               = 50.25 MiB   per sequence
x 4 sequences                              = 201 MiB = 0.196 GiB

That last line is the one most budgets for this model get wrong, and it is worth following because it also explains why the KV cache is not multiplied the same way — and because the same default is what Step 3's --parallel warning turns on. common/arg.cpp sets params.n_parallel = -1 for the server example specifically; tools/server/server.cpp resolves that sentinel with params.n_parallel = 4; params.kv_unified = true;; common/common.cpp then assigns cparams.n_seq_max = params.n_parallel. src/llama-model.cpp sizes the recurrent memory with std::max((uint32_t) 1, cparams.n_seq_max) on the branch that names LLM_ARCH_QWEN35 explicitly, and src/llama-memory-recurrent.cpp allocates n_rows = mem_size * (1 + n_rs_seq), where n_rs_seq defaults to 0. So the recurrent state is allocated once per slot — four times. The same default line sets kv_unified = true, which gives all four slots one shared attention cache, so the 32 KiB/token figure above is not multiplied. One term scales with the slot count and the other does not; a budget that treats them alike is wrong in one direction or the other.

The accountable ladder. BF16 text weights + f16 projector + KV + recurrent state, each cell summed from the exact byte-derived inputs and rounded once:

Context	Weights	KV (f16)	Recurrent state	Accountable
32,768	17.545	1.000	0.196	18.741
65,536	17.545	2.000	0.196	19.741
131,072	17.545	4.000	0.196	21.741
262,144 (native)	17.545	8.000	0.196	25.741

The term this card cannot leave out of the table

On every other card in this catalogue that is the whole budget, because compute buffers are small and un-derived. Here one of them is neither, and it is a direct consequence of the --flash-attn off in Step 3.

Flash attention exists partly so that the n_kv × n_tokens × n_head matrix of attention scores never has to exist in memory. Turn it off and llama.cpp takes the explicit branch in src/llama-graph.cpp, which builds kq = ggml_mul_mat(ctx0, k, q) and immediately pins its precision: ggml_mul_mat_set_prec(kq, GGML_PREC_F32), with the source's own reason alongside it — "this op tends to require high floating point range". So the graph allocator must reserve, for one layer at a time, four bytes per score:

n_kv (= -c)  x  n_ubatch (512 default)  x  n_head (16)  x  4 B (F32)
65,536       x  512                     x  16           x  4  = 2.000 GiB

n_ubatch is 512 by default (common/common.h) and n_head is this model's num_attention_heads: 16 — a value the 9B shares with the 4B, since the extra parameters went into intermediate_size, not into more heads. The term is linear in -c, exactly like the KV cache, so on this card every context doubling costs twice what the table above says:

Context	Accountable	F32 scores (flash attention off)	Real footprint	Left of 32 GiB
32,768	18.741	1.000	19.741	12.3
65,536	19.741	2.000	21.741	10.3
131,072	21.741	4.000	25.741	6.3
262,144 (native)	25.741	8.000	33.741	— does not fit

Three things to read off that table rather than discover. First, the native 262,144-token window does not fit even here — 33.741 GiB against 32 — which is the one hard limit on this page. Second, in this flash-attention-off configuration 131,072 fits only on this card: at 25.741 GiB it is over a 24 GiB card outright, so it is a genuine 32 GB affordance rather than a nicety. Third, the row this recipe leads costs 21.741 GiB, which is where min_vram_gb: 24 comes from.

Why 65,536 tokens and not 131,072

131,072 fits with 6.3 GiB spare, and the 24 GB recipes for the 4B in this catalogue lead it, on an argument verified in the harness source: the per-round user text is bounded and tiny (the URL line is trimmed to MAX_URL_LENGTH = 100, the standing instruction is one fixed sentence, and the post-action observation is capped at max_observation_chars: int = 1000 and only produced for one action type — _TEXT_OBSERVATION_ACTIONS is the single-element frozenset {"read_page_answer_question"}), so the only genuinely unbounded term is the model's own reasoning plus tool call, once per round. The image half is capped outright at max_n_images: int = 3 in fara15_agent.py, which at the derived 1,260 tokens per screenshot pins images at 3,780 tokens for a whole run.

That argument about the harness is unchanged at 9B. What changed is the price. fara-cli defaults to --max_rounds 100 (run_fara.py), so:

 65,536 - 3,780 image tokens =  61,756 for text / 100 rounds ~   618 tokens per round
131,072 - 3,780             = 127,292 for text / 100 rounds ~ 1,270 tokens per round

On the 4B, buying the second row cost 2.000 GiB. Here it costs 4.000 GiB — 2.000 of KV and 2.000 more of F32 scores, because flash attention is off — for a window whose only consumer is the model's own reasoning, on a hundred-round default the harness does not raise for you. That is the whole of this page's remaining margin over a 24 GB card, spent on a term nothing in the loop fills. So the default here is 65,536, still more than four times the harness's documented 15,000-token floor.

The honest counterweight, since Fara emits a chain-of-thought block before every action: 618 tokens per round is a budget, not a guarantee, and context shift is off by default (ctx_shift = false in common/common.h), so an overrun is a hard error rather than a silent truncation of the oldest turns. If you raise --max_rounds, or you watch a run and see long <think> traces, -c 131072 is a one-word change costing 4.000 GiB and, with flash attention off, this is the only card in the catalogue on which it still fits. Note also that halving -ub halves the scores term at no cost to trajectory length — -c 131072 -ub 256 lands at 23.741 GiB, cheaper than the full-batch 65,536 row plus a doubled window. Size the window for the run you intend.

Why BF16 rather than a quant — and one extra reason on this silicon

Fara's output is pixel coordinates — the card describes its grounding data as "Curated datasets for predicting actions and pixel coordinates from screenshots, with images, text, and bounding boxes." Quantisation error lands on exactly that faculty, and no published evaluation measures how much grounding accuracy any tier costs. The vendor's published scores are for BF16. On the smaller cards in this catalogue there is no choice; here the question disappears.

BF16 is a first-class type on this backend rather than a tolerated one, which is worth establishing before you download nineteen gigabytes of it:

• The hardware predicate. bf16_mma_hardware_available() in ggml/src/ggml-cuda/common.cuh returns true for GGML_CUDA_CC_IS_NVIDIA(cc) && cc >= GGML_CUDA_CC_AMPERE, i.e. compute capability 800 and up. The same file defines GGML_CUDA_CC_BLACKWELL as 1200. This card clears the runtime's own BF16 tensor-core check by a wide margin.
• The decode kernel. ggml/src/ggml-cuda/mmvf.cu has explicit case GGML_TYPE_BF16 branches and reads weights through nv_bfloat162 pairs. Single-token generation from BF16 weights runs a purpose-built matrix-vector kernel; it is not converted to F16 first.
• The batched path. Prompt processing dispatches ggml_cuda_mul_mat_cublas_impl<GGML_TYPE_BF16> in ggml/src/ggml-cuda/ggml-cuda.cu, whose traits map BF16 to CUDA_R_16BF with F32 accumulation.

And there is a Blackwell-only reason that does not apply to the Ampere or Ada cards in this catalogue. llama.cpp's quantised matrix-multiply path (MMQ) is the subject of an open bug whose title names this card — #23385 — and a BF16 model never enters that path: ggml_cuda_should_use_mmq in mmq.cu enumerates the quantised types it accepts and returns false by default for everything else, so BF16 dispatches through mul_mat_vec_f and cuBLAS instead. Leading with unquantised weights removes a whole class of Blackwell failure from the picture rather than working around it. Troubleshooting has the detail.

What BF16 costs, stated plainly. The BF16 file is 1.877x the bytes of the Q8_0 one (17,920,697,216 / 9,545,983,104), and single-token decode on a model this size is bound by how many weight bytes leave memory per token — so this is the slower configuration. Nobody has measured Fara1.5 on any consumer GPU, so this recipe publishes a direction and not a ratio. If bandwidth matters more to you than the open question about quantised grounding, the same repo's ladder is a one-word substitution in Step 2 and Step 3 — accountable totals at the 65,536-token window: Fara1.5-9B-Q8_0.gguf 8.890 GiB of weights, 11.942 total; Q6_K 7.171 / 10.223; Q5_K_M 6.382 / 9.434; Q4_K_M 5.505 / 8.556 — but on this card that also puts you back inside the MMQ path above, which is the trade showing up.

Why the f16 projector next to BF16 text weights. This deserves an actual answer rather than a habit, because the argument that picks BF16 for the text weights — run the dtype the vendor evaluated — points straight at mmproj-bf16, and this recipe does not take it. Start from which build is which, because config.json will mislead you here: it declares "dtype": "bfloat16" under text_config and "dtype": "float32" under vision_config, but that field records an intent, not the contents of the repo. Read the safetensors headers instead and the vision tower ships exactly like the rest of the model — 333 vision tensors, every one BF16. So mmproj-Fara1.5-9B-bf16.gguf is the byte-faithful conversion and mmproj-Fara1.5-9B-f16.gguf is a re-encode, and taking the re-encode is a deliberate trade rather than a free one. Two things buy it. Precision at equal width: IEEE binary16 spends three more mantissa bits than bfloat16 over a narrower exponent range, and a vision tower's activations are range-bounded — the finer grid is worth more than the headroom on the stage that turns patch embeddings into the coordinates this model clicks on. And comparability: every accountable total on this page, and on every other Fara1.5 recipe in this catalogue, is computed with the f16 projector, so the budgets line up card to card. Neither of those is a measurement. What does not enter the decision on this card is the MMQ path above: neither projector build is a quantised type, so ggml_cuda_should_use_mmq returns false for both and the choice carries no Blackwell risk either way. If you would rather run the vendor's bytes, mmproj-Fara1.5-9B-bf16.gguf is a one-word substitution costing 3,538,944 B — 3.375 MiB, which moves the 65,536 accountable total from 19.741 to 19.745. Nobody has measured which grounds better.

Nothing in the tables above is free space you can plan against to the last byte: the CUDA context, the remaining compute buffers and the vision encoder's working set are allocated at load time and this recipe does not derive them. Read the CUDA0 compute buffer size lines llama-server prints on startup for the real figures on your build, watch nvidia-smi with the agent actually running, and report what you see.

How the context is actually spent

A CUA sends a screenshot every step, so the per-image cost decides whether the budget above is realistic. It is not the number implied by the projector's position-embedding grid: llama.cpp treats this projector as dynamic-resolution, and --image-min-tokens / --image-max-tokens are documented as "only used by vision models with dynamic resolution".

The harness fixes the browser viewport at 1440x900 (viewport_width: int = 1440, viewport_height: int = 900 in fara15_agent.py), matching the card's note that "The screen resolution Fara is most commonly trained with is" 1440x900. It pre-resizes with Qwen's smart_resize at patch size 16 and merge size 2, so the factor is 32 and 900 becomes 896. llama.cpp then counts merged patches per axis in tools/mtmd/clip.cpp, where the PROJECTOR_TYPE_QWEN3VL branch returns (img->nx() / params.patch_size) / 2 across and (img->ny() / params.patch_size) / 2 down:

across : (1440 / 16) / 2 = 90 / 2 = 45
down   : ( 896 / 16) / 2 = 56 / 2 = 28
tokens : 45 x 28                 = 1,260 per screenshot
x max_n_images = 3               = 3,780 tokens resident

The source config.json's vision_config agrees on the geometry (patch_size: 16, spatial_merge_size: 2, depth: 27, out_hidden_size: 4096). The harness's own conservative default is image_token_estimate: int = 1500, a useful sanity check on the arithmetic, and the model card agrees on the retention: "We only keep the most recent 3 screenshots" in the chat history.

One consequence matters for the crash risk on this silicon, and it is easy to miss: because the image history is capped but the text history is not, the number of text tokens sitting in front of each screenshot grows with every round. That is one of the variables #25717 identifies as a trigger. Step 5's second test probes it deliberately.

Installation

1. Build llama.cpp with CUDA 12.8+ for sm_120

This is the one step that genuinely differs on Blackwell. The RTX 5090 is compute capability 12.0, and llama.cpp's CUDA CMakeLists annotates that target as "Blackwell, needs CUDA v12.8, FP4 tensor cores". Check the toolkit first, because this is the only reliable way to check it:

nvcc --version    # must report release 12.8 or newer

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

Two details worth knowing rather than pasting:

• Pass 120 explicitly — but not because an older toolkit would silently skip your card. It would not: the default path never reaches the architecture list at all. ggml/src/ggml-cuda/CMakeLists.txt takes set(CMAKE_CUDA_ARCHITECTURES "native") whenever GGML_NATIVE is on, and ggml/CMakeLists.txt sets GGML_NATIVE_DEFAULT ON unless you are cross-compiling — so on an ordinary build an inadequate toolkit fails loudly. Two real reasons to name the architecture anyway: it makes the build reproducible rather than dependent on which GPU happened to be visible to CMake, and it closes the native path's one soft failure — docs/build.md documents nvcc warning : Cannot find valid GPU for '-arch=native', default arch is used, which is a warning you can scroll past on the way to a binary that does not target your card.
• You write 120, the build system compiles 120a. The same file rewrites any plain 12X architecture into its architecture-specific 12Xa form because, in its own words, "the Blackwell FP4 tensor core instructions are not forwards compatible". You do not type the a, and you should not be surprised to see 120a-real in the CMake status line. Do not read that line as a toolkit check — Troubleshooting explains why it proves nothing.

Build from a release at or after b10087 — the release the GGUF used here was quantized on. That is also newer than the builds #25717 was filed against, which is one reason to test rather than assume the report reproduces.

Both halves of the model are first-class in mainline — worth confirming before you download 19 GB:

grep -n 'LLM_ARCH_QWEN35,' src/llama-arch.cpp        # -> { LLM_ARCH_QWEN35, "qwen35" }
grep -n 'qwen3vl_merger' tools/mtmd/clip-impl.h      # -> PROJECTOR_TYPE_QWEN3VL

A note on FP4, since Blackwell is why people ask. The FP4 tensor cores in this silicon are real and llama.cpp does use them: GGML_TYPE_NVFP4 is a shipped quant type in ggml/include/ggml.h, with a native Blackwell path in mmq.cu. What is missing is not the loader — it is an artifact: searching the Hub for Fara1.5 returns no NVFP4 build of the 9B in any org. And FP4 is a 4-bit tier, i.e. the opposite direction from what this card's 32 GB is for. Build for 120 because that is your architecture, not because it unlocks a 4-bit path.

2. Download the weights and the projector

Both files are required. The projector is a separate artifact — downloading only the text GGUF gives you a model that cannot see.

pip install -U huggingface_hub

hf download bartowski/Fara1.5-9B-GGUF \
  Fara1.5-9B-bf16.gguf mmproj-Fara1.5-9B-f16.gguf \
  --local-dir ~/models/fara1.5-9b

That is 18.84 GB over the wire, so start it before you read the rest of the page.

Microsoft publishes no first-party GGUF for Fara1.5 — the microsoft org ships Fara1.5-4B, Fara1.5-9B and Fara1.5-27B as safetensors only. Four independent community quantisers have converted the 9B including its vision tower — bartowski, prithivMLmods, DevQuasar and Tdamre each publish a matching mmproj. Four independent conversions succeeding is the strongest available evidence that the vision path converts cleanly; it is still not a runtime test.

3. Serve it

./build/bin/llama-server \
  -m ~/models/fara1.5-9b/Fara1.5-9B-bf16.gguf \
  --mmproj ~/models/fara1.5-9b/mmproj-Fara1.5-9B-f16.gguf \
  -ngl 99 -c 65536 --flash-attn off \
  --host 127.0.0.1 --port 5000 \
  --image-min-tokens 1024 \
  --reasoning-format none \
  --temp 0

Four of those flags are load-bearing and none is a default:

• --flash-attn off — the one that is specific to this silicon, and the reason it is spelled out rather than left alone. The flag is documented in common/arg.cpp as [on|off|auto] with the struct default LLAMA_FLASH_ATTN_TYPE_AUTO, and src/llama-context.cpp sets cparams.flash_attn = params.flash_attn_type != LLAMA_FLASH_ATTN_TYPE_DISABLED, so AUTO starts out enabled and is then confirmed by a runtime probe. Flash attention on is the strongest trigger in #25717's crash matrix — the only variable that crashed on its own, with a short prompt. Passing nothing here would document the crashing configuration. It is not protection, and it is not free. Two rows of that matrix crash with it off, so read it as removing a necessary condition rather than as a fix; and see the F32 scores term above, which is what it costs.
• --reasoning-format none — mandatory. Without it the agent loop silently receives nothing; Troubleshooting has the mechanism.
• --image-min-tokens 1024 — this projector ships no minimum-pixel key, so llama.cpp falls back to the floor set by hparams.set_limit_image_tokens(8, 4096) in the PROJECTOR_TYPE_QWEN3VL branch of tools/mtmd/clip.cpp and warns, a few lines later in the same branch: "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks". Fara is a grounding model, so that warning is aimed precisely at this case. At the harness's viewport the image is already 1,260 tokens and the floor never binds; it protects you if you shrink the viewport.
• -c 65536 — see the budget tables and the window discussion above. This is a deliberate choice sized to --max_rounds 100, not the model's native maximum, which does not fit even this card.

And one flag to deliberately not pass: --parallel. This is the other half of the #25717 mitigation, and it is a subtraction rather than an addition, which makes it easy to get wrong in both directions.

The two configurations in that crash matrix which completed both ran a unified KV cache — and on llama-server that is already the default, so you get it by leaving the flag alone: the n_parallel = -1 sentinel resolved by server.cpp into n_parallel = 4; kv_unified = true; is the same line the recurrent-state derivation above hangs on. Passing --kv-unified yourself is therefore a no-op here, which is why it is absent from the command above — its own help text says as much: "use single unified KV buffer shared across all sequences (default: enabled if number of slots is auto)". It would also be the wrong lesson to draw from the report, since one of the crashing rows passes --kv-unified explicitly.

What matters is the opposite move. Passing any explicit --parallel N takes n_parallel out of the auto sentinel, so the server block never runs, kv_unified stays at its false struct default, and src/llama-context.cpp then computes n_ctx_seq = n_ctx / n_seq_max — splitting the cache, which is one of the report's three triggers. Both flash-attention-off crashes in the matrix passed --parallel. On this page's window the split is survivable in size terms (--parallel 4 on 65,536 leaves each slot 16,384, still above the harness's 15,000-token floor — but only just), so the reason to avoid it here is the crash correlation, and the context floor is a second, thinner reason rather than the main one.

4. Confirm what actually loaded

./build/bin/llama-server --version    # build number >= 10087

Four lines of the startup log are worth reading before going further: the CUDA0 device line (it should name your 5090 — if the model loaded on CPU, go back to Step 1), the flash_attn line (it must say disabled; if it does not, your flag did not take), the clip_ctx: CLIP using <backend> backend line — clip.cpp emits it as "CLIP using %s backend" with the backend's own name, and CPU there means the projector is not on the GPU — and the mmproj load stage itself.

5. Prove the vision path works — twice, and the second time is the one that matters

Check that the server advertises multimodal support, then send a real image. The server docs make the first part a client-side requirement rather than an optional nicety: "Clients should check /models or /v1/models for the multimodal capability before a multimodal request." Run GET /props alongside it — it reports which modalities loaded rather than merely that some did, the more useful signal when a projector silently failed.

curl -s http://127.0.0.1:5000/v1/models | grep -o multimodal

# supplementary — names the modality that actually loaded
curl -s http://127.0.0.1:5000/props | python3 -c "import sys,json; print(json.load(sys.stdin)['modalities'])"
# expect: {'vision': True}

# test A — short prompt, image first
curl -s http://127.0.0.1:5000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "fara",
  "temperature": 0,
  "messages": [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}},
    {"type": "text", "text": "Describe this image in one sentence."}
  ]}]
}' | python3 -m json.tool

Test A is not sufficient on this silicon. In #25717's matrix a short prompt with flash attention off completed, and the same configuration with roughly 800 text tokens in front of the image did not — and a real agent prompt is the long one. So run the second test before you trust the setup:

# test B — ~800 tokens of text BEFORE the image, which is the agent-run shape
python3 - <<'PY'
import json, urllib.request
long_text = "Describe precisely. " * 200
body = {"model": "fara", "temperature": 0, "max_tokens": 200,
        "messages": [{"role": "user", "content": [
            {"type": "text", "text": long_text},
            {"type": "image_url", "image_url": {"url":
             "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}]}]}
req = urllib.request.Request("http://127.0.0.1:5000/v1/chat/completions",
        data=json.dumps(body).encode(), headers={"Content-Type": "application/json"})
print(urllib.request.urlopen(req, timeout=300).read()[:400])
PY

If both replies describe the picture, you have cleared the risk this page is most worried about. If either fails silently — the model answering as though no image were present — stop; the agent loop below will produce confidently wrong actions rather than fail loudly. If test B kills the server, watch the log for a find_slot: non-consecutive token position line immediately before the CUDA error and go to Troubleshooting: that would be the reported failure reproduced on a new model, a different Blackwell card and a newer build, which is worth a comment on the upstream issue.

6. Install the harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Skip the README's [vllm] extra — that path serves BF16 safetensors on datacenter cards, which is not how this recipe gets to BF16.

Running

Two clients share the same llama-server endpoint. Path A (fara-cli) is the smoke test, with no isolation. Path B (Magentic-UI) is what the vendor recommends for anything real. They are not equivalent in risk; pick deliberately.

Path A — fara-cli, for a first smoke test

fara-cli speaks to any OpenAI-compatible endpoint, and this is the one size where its built-in defaults already match: DEFAULT_ENDPOINT_CONFIG in run_fara.py is {"model": "Fara1.5-9B", "base_url": "http://localhost:5000/v1", ...}. Name them explicitly anyway, so the command does not depend on a default that could move:

source .venv/bin/activate
fara-cli \
  --base_url http://127.0.0.1:5000/v1 \
  --api_key sk-no-key-required \
  --model fara \
  --task "find the current UTC time on time.is and tell me" \
  --output_folder ./runs/first \
  --save_screenshots \
  --headful \
  --max_rounds 15

--headful is in that command on purpose. The harness runs the browser headless by default — run_fara.py declares the flag with the help text "Run the browser in headful mode (show GUI, default is headless)" and both launch sites read headless=not args.headful. Either way this drives a real, unisolated Chromium on your machine; the danger is identical with or without the flag. What the flag changes is whether you can see it, and the supervision advice above assumes you can — so it is what makes that advice true rather than aspirational.

Each step writes a screenshot and a data_point.json into --output_folder, which is how you tell a grounding failure (clicks landing in the wrong place) from a reasoning failure. Start with the low --max_rounds 15 above for a second reason on this card: if the prompt-cache growth in the report cited in Troubleshooting is real on a source build, it shows up on long runs, not short ones.

Keep the task on a low-stakes public site and stay signed out. Fara is trained to stop and ask before entering personal information, making payments, submitting forms, signing in or sending messages; if you see it sail through one of those, stop the run — that is a wiring problem in your prompt path, not a quirk to tolerate.

Path B — Magentic-UI, for the container

Everything the model card assumes — the containerised browser, domain allow-lists, watch-mode and a pause control — lives in Magentic-UI, not in fara-cli. That is also where a Docker requirement would come from; there is no Docker step in Path A because Path A genuinely does not use one.

Magentic-UI takes the same OpenAI-compatible endpoint you started above, so nothing on this page changes except the client. Its setup is documented in its own repository rather than reproduced here, because this recipe has not verified those steps end to end on this GPU — give it http://127.0.0.1:5000/v1 as the model endpoint. Whichever client you use, match the trained resolution: the card is explicit — "Match this in your sandbox for the most reliable grounding."

Results

• Speed: omitted — no published measurement of Fara1.5-9B exists for this card or for any consumer GPU, and the only hardware Microsoft names is datacenter class. Borrowing a figure from an A100 for a 32 GB Blackwell card would be misleading, and the vendor publishes no throughput numbers to borrow in the first place. If you run this, please contribute your numbers so this line can be replaced with a measurement — and if you run both weight tiers, the BF16-versus-Q8_0 step latency is the most useful pair of numbers anyone could add to this page.
• VRAM usage: 17.545 GiB of weights and projector; 19.741 GiB accountable including the f16 KV cache and the four-slot recurrent state at a 65,536-token context, plus a bounded 2.000 GiB of F32 attention scores because this card's configuration disables flash attention — 21.741 GiB in total, and the reason min_vram_gb is 24. All of it is derived above from byte counts and llama.cpp's memory formulas, not observed; the vision graph, the remaining compute buffers and the browser's own GPU memory still sit on top. Track the empirical picture at /check/fara1-5-9b/rtx-5090.
• Quality notes: the vendor's published scores for this size — 86.6 WebVoyager, 63.4 Online-Mind2Web, 32.3 WebTailBench, per the model card's evaluation table — are for the BF16 model on datacenter hardware. The hardware does not transfer; the precision does, which is the point of leading BF16 here. No public evaluation measures what quantisation costs a model whose job is predicting exact pixel coordinates, and on this card you do not have to find out.
• What Blackwell changes, and what it does not. Nothing about this model is CUDA-generation-dependent by design — the architecture is fully registered in mainline llama.cpp and the multimodal layer contains no architecture-specific code. What sm_120 changes is the maturity of the kernels and drivers underneath, which is what the first three Troubleshooting entries are about.
• Language: English only — the card puts "Languages other than English (training data is English-only)" out of scope.

For the full benchmark data, see /check/fara1-5-9b/rtx-5090.

Troubleshooting

The server dies on an image request with CUDA error: an illegal memory access was encountered

This is the failure to know about on sm_120, and it is why Step 5 has two tests. llama.cpp #25717 reports llama-server crashing with exactly that error on vision requests, preceded in the log by find_slot: non-consecutive token position. The reporter bisected seven configurations; the triggers are flash attention, a split KV cache, and — the surprising one — the amount of text placed before the image in the prompt. Two rows completed: flash attention off with the default unified cache and a short text prefix, and one where the same long prompt was sent with the image as the first content part.

Read it with its limits stated, because they matter and because the mitigation is shaped by them rather than by the headline:

• The architecture family is this one and the silicon family is this one, but the card is not. The report's own hardware line is an RTX 5070 Ti 16 GB on sm_120. Nobody has filed the same failure against this card.
• It is a community report with no maintainer response. One comment exists and it is the reporter's own follow-up; the filer has no association with the project, and no pull request references it.
• The model is not this one. It was filed against a 35B-A3B mixture-of-experts member of the same family — a different architecture enum with a different layer count and KV-head count — quantised, with expert CPU offload (--n-cpu-moe 24) and a 4096-token physical batch in play, on builds older than the one Step 1 produces. None of that MoE configuration transfers, and a 9B dense BF16 model at the default -ub 512 exercises a materially simpler memory plan.
• The implicated mechanism does transfer, and that is checkable rather than assumed, in two independent places. The source config.json sets rope_parameters.mrope_interleaved: true with mrope_section: [11, 11, 10] — this model uses the same M-RoPE positioning the report blames — and it loads the same qwen3vl_merger projector. Separately, the non-consecutive token position line that precedes the crash is emitted from src/llama-memory-recurrent.cpp, i.e. from the recurrent half of the hybrid cache, which 24 of this model's 32 layers traverse. That is not MoE-specific.

So: a real, specific risk worth shaping the command around, not a demonstrated failure of this configuration. What this recipe does about it is cheap — --flash-attn off plus not passing --parallel reproduces the layout of the rows that completed, and Step 5's test B probes the long-text-prefix trigger in ten seconds. Do not read --flash-attn off as protection: two rows of the matrix crash with it off, so it removes a necessary condition and no more. Its one real cost is memory rather than throughput, and it is the term that sets this page's floor.

One piece of luck is on your side, and it is worth more than the flags: the harness builds every user turn with the screenshot as the first content part and the text after it — content=[ImageObj.from_pil(scaled_screenshot), text_prompt] in fara15_agent.py, at both call sites — which is the ordering of the only long-prompt row that completed. Fara's system prompt is still long text ahead of the first image in the flattened sequence, and that prefix grows with every round, so the shape is not exactly either reported case. If test B crashes and test A does not, you have reproduced the report on a new model, a different card and a newer build — post it.

A fatal abort in the MMQ kernels, or a CUDA invalid-value at kernel launch

llama.cpp #23385 is open and names this card in its own title — "Eval bug: Fatal MMQ crashes on Blackwell (RTX 5090/5080) due to unhandled sharedMemPerBlockOptin driver bug" — with a model line of "Any model". The root cause is a driver defect rather than a llama.cpp one: early Blackwell drivers return a corrupted cudaDeviceProp.sharedMemPerBlockOptin, and that value flows into the shared-memory budget the quantised matmul kernels dispatch against. mmq.cu has since added a floor ("MMQ tiles require at least 48 KiB per-block shared memory; fall back to BLAS otherwise.") which turns the zero case into a cuBLAS fallback rather than an abort; the overflowed variant clears that floor trivially and is not covered by it. The issue is community-filed with three comments and no maintainer fix merged for the MMQ path.

Three ways out, in order: update the graphics driver (the root cause is the driver); rebuild with -DGGML_CUDA_FORCE_CUBLAS=ON, a real option in ggml/CMakeLists.txt described as "ggml: always use cuBLAS instead of mmq kernels"; or run a non-quantised model, which is what this recipe leads with — ggml_cuda_should_use_mmq accepts only quantised weight types, so BF16 never reaches that code at all. If you switched to Q8_0 for bandwidth and then hit this, that is the trade showing up.

The build succeeds but the GPU is idle, or the server falls back to CPU

Check the toolkit directly first: nvcc --version must report 12.8 or newer. Blackwell's FP4 tensor-core instructions are not forwards-compatible, so an older toolkit cannot emit sm_120a code at all.

Be aware, though, that a too-old toolkit is the least likely explanation for an idle GPU, because it normally fails the build outright rather than producing a working binary with no kernels for your card (Step 1 has the mechanism). The two explanations that actually reach a running server are a -arch=native build where CMake could not see the GPU — grep the build log for the Cannot find valid GPU warning docs/build.md documents — and a binary you did not compile yourself, below.

Do not try to confirm the toolkit from the configure log. The 12X → 12Xa rewrite in the CUDA CMakeLists is unconditional — it sits outside the if (NOT DEFINED CMAKE_CUDA_ARCHITECTURES) block that holds the toolkit-version gate, and Step 1 passes -DCMAKE_CUDA_ARCHITECTURES=120 explicitly, which skips that block entirely. So Replacing 120 … with 120a prints on every toolkit version and proves nothing. nvcc --version is the test.

Relatedly: build from source rather than reaching for a third-party "sm120 Blackwell" binary. #26205 is an open report of llama-server crashing silently before it starts listening on an sm_120 card under exactly that kind of prebuilt Windows build, with an official older-CUDA build giving 0% GPU utilisation on the same machine — the two failure modes this step exists to prevent, in one report. Its hardware line names a different Blackwell card, so read it as an argument for building rather than as a report about this one.

The agent does nothing — every step logs an empty response or list index out of range

This is the failure everyone hits, and it is not a bug in your setup. Fara opens a <think> block and never closes it — you can read that straight out of the chat template embedded in the GGUF you downloaded, whose generation prompt emits '<think>\n' with no closing tag unless enable_thinking is explicitly false. Under llama-server's default reasoning extraction the parser therefore treats the whole output — reasoning and the <tool_call> block together — as reasoning, returns it in reasoning_content, and leaves message.content empty. The harness parses message.content for <tool_call>, finds nothing, and the loop dies on round one.

Launch with --reasoning-format none, documented as "none: leaves thoughts unparsed in message.content". The raw output then begins with a stray unclosed <think> tag, which is cosmetic.

Reported for the 27B sibling by a community user in the model family's HF discussions and acknowledged there by Microsoft — swhitehead, whose HF account carries the org-member flag on that repo, replied "Would you mind posting this as an issue on the Fara1.5 github repo so we can mention this in the hosting instructions?", which became microsoft/fara#82 — still open at the time of writing, titled "Hosting: llama.cpp needs --reasoning-format none". It applies to the 9B for a checkable reason rather than by assumption: the 9B's own GGUF carries the same chat template, whose generation prompt emits the unclosed <think>. The model card documents the output shape it comes from — "Chain-of-thought block followed by a tool-call block (XML-tagged)".

The model answers but ignores the screenshot

Almost always the projector. Confirm --mmproj is on the command line, that the server logged an mmproj load stage and a CLIP using … line, that /v1/models advertises the multimodal capability, and that Step 5's /props call reports vision: True. A text-only GGUF with no projector loads and runs perfectly happily — it simply cannot see, which in a CUA presents as confidently wrong actions rather than an error.

Clicks land in the wrong place

Check the viewport first: the harness defaults to 1440x900 because that is what the model was mostly trained at, and a different window size changes both the grounding distribution and the token cost derived above. Then confirm --image-min-tokens 1024 is set, since llama.cpp's own warning ties sub-1024-token images to degraded grounding. Only after both should you look at the weights — and on BF16 there is nothing above you to climb to, which is exactly why this configuration is a good place to debug grounding from.

A long agent run climbs in system RAM, or the server dies after many steps

llama-server's prompt cache and context checkpoints are sized for chat, not for a stream of unique screenshots: the defaults are cache_ram_mib = 8192 and n_ctx_checkpoints = 32 per common/common.h. #26216 reports the cache growing past its own cap and a deterministic segfault under sustained unique-prompt vision load — the exact shape of a long CUA run. Three caveats before you treat it as a property of your build: it was closed as not_planned within fifteen minutes and drew no discussion, it was reproduced on the project's container images rather than a source build, and its host line names a different Blackwell card than this one. The knobs are cheap regardless: cap the cache with -cram 2048 (or -cram 0 to disable it) and reduce -ctxcp if a many-round run climbs. This is a system-RAM problem, not a VRAM one — 32 GB of VRAM does not help.

Out of memory at load

21.741 GiB of real footprint against 32 GiB gives you room, but the levers here are not the ones the smaller-card recipes reach for, because the F32 scores term derived above responds to a knob the others never touch.

Cheapest first: halve -ub. The scores tensor is n_kv × n_ubatch × n_head × 4 B, so it is linear in the physical batch size as well as in the context. -ub 256 takes it from 2.000 GiB to 1.000 GiB, and -ub 128 to 0.500 GiB, at the cost of slower prompt processing — which for a CUA means a slower image encode once per step, not slower generation. Unlike halving the window it costs you nothing in trajectory length, which is why it goes first.

Then -c: 65536 → 32768 frees 1.000 GiB of KV and 1.000 GiB of scores, 2.000 GiB in total, and is still more than twice the harness's 15,000-token floor.

Do not reach for the usual --cache-type-k q8_0 --cache-type-v q8_0 here — the V half is incompatible with the --flash-attn off this card's configuration needs. llama-context.cpp rejects that combination outright: "quantized V cache requires flash_attn to be enabled", returning a null context rather than starting. Quantising the K half alone is legal and needs no flash attention: block_q8_0 is one ggml_half scale plus 32 int8_t quants, 34 bytes per 32 values against f16's 64, so the K term drops from 16 to 8.5 KiB/token and the total from 32 to 24.5 KiB/token — 1.531 GiB instead of 2.000 GiB at a 65,536-token context. Keep the f16 projector regardless: at 0.855 GiB it is not where the savings are, and the q8_0 projector some quantisers ship trades vision fidelity in a model whose entire job is looking at pixels. If you need it out of VRAM entirely, --no-mmproj-offload keeps it on the CPU at the cost of slower image encoding — once per agent turn. Take that flag as a memory lever and nothing more: row 3 of #25717's own matrix ran --no-mmproj-offload with flash attention on and crashed anyway, so moving the encoder off the GPU frees VRAM without dodging the bug above.

Don't reach for speculative decoding to fill the headroom

The model family carries a multi-token-prediction layer — the source config.json declares mtp_num_hidden_layers: 1 inside text_config. That layer is not part of the GGUF conversion path used here: llama.cpp names the corresponding header key nextn_predict_layers (LLM_KV_NEXTN_PREDICT_LAYERS in src/llama-arch.cpp), and the file this recipe downloads carries no blk.*.nextn.* tensors for --spec-type to attach to. Nothing on this page enables it; if you go looking, that is why it does nothing.

rope.dimension_sections has wrong array length; expected 4, got 3

This is a real open llama.cpp bug (PR #25334, titled "fix: load qwen35 Ollama GGUF exports"), but it belongs to Ollama-exported qwen35 blobs, not to the GGUF this recipe uses. The source config's rope_parameters.mrope_section is [11, 11, 10], which the bartowski conversion writes with the trailing zero llama.cpp expects. If you see this error, you are loading a different file than the one downloaded in Step 2. Relatedly, ollama run fara does nothing: there is no Ollama library entry for Fara1.5 in any form. The GGUF / LM Studio / Ollama line in the harness README refers to Fara-7B, the previous generation, reached with a --fara-7b flag.

Generation is slower than you expected

Two things to separate before you go bug-hunting. The first is expected: BF16 moves 1.877x the bytes per token that Q8_0 does, and on a 9B that is nearly eight gigabytes of extra traffic per token-generation pass — a cost you chose in exchange for skipping the quantisation question. A/B against Fara1.5-9B-Q8_0.gguf with llama-bench before treating it as a defect, remembering that Q8_0 puts you back inside the MMQ path above. The second is a lead rather than a known property: llama.cpp issue #25162 is an open, community-reported regression against exactly the SSM/linear-attention kernels this architecture leans on, titled "Performance regression on Turing GPUs after 9e58d4d69 — Qwen35 SSM kernels affected". Read its scope carefully: the title says Turing, the reporter measured on an RTX 2080 Ti, the thread's two comments are both from community accounts with no maintainer response, and nobody has posted numbers for a Blackwell card. It is a thing to check with llama-bench across a couple of builds, not a property of this one.