How much VRAM does Fara1.5-27B need?

About 32 GB — the minimum this recipe targets.

How hard is this setup?

Advanced — follow the steps above.

Fara1.5-27B on RTX 5090: Q6_K Computer-Use Agent on Blackwell sm_120

What You'll Build

A local browser computer-use agent (CUA): Microsoft's Fara1.5-27B served by llama-server on an RTX 5090, with the vision projector loaded so the model actually sees browser screenshots, driven by Microsoft's own fara-cli harness against a plain local Playwright Chromium.

Two things make this page different from the 24 GB recipes for the same model. First, 32 GB buys two quant tiers: the 24 GB cards in this catalogue lead Q4_K_M, and this one leads Q6_K at the same context window — which matters because Fara emits pixel coordinates and nobody has measured what quantisation costs that faculty. Second, this is the one architecture in the sweep with an open report of llama-server crashing on vision requests specifically on sm_120; mitigating it means turning flash attention off, and that costs memory scaling with your context. The extra 8 GB pays for both, and this page shows the arithmetic for both.

Hardware data: RTX 5090 (32GB VRAM) · Q6_K weights + f16 mmproj = 22.493 GiB · 25.078 GiB accountable at a 32,768-token context, 26.578 GiB including the F32 attention scores this card's flash-attention-off configuration adds (both derived below) · See benchmark data

⚠️ This model clicks, types and submits in a real browser, and the path documented here gives it no isolation of any kind. That is the operating requirement, not boilerplate, and it is the thing to settle before anything else on this page.

The model card is blunt about the consequence: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." That sentence closes the card's Recommended Deployment: MagenticLite section, which lists four controls as MagenticLite features — the first of them sandboxing, described there as "the browser runs in a Docker container with no access to host files or environment variables" — and then says: "If you integrate Fara1.5-27B directly, you're responsible for these controls." The card's Responsible AI section asks you to "run Fara in an isolated container with no access to host files, environment variables, or sensitive credentials". Fara also reads whatever is on the page, so a hostile page is a prompt-injection surface — the card lists "Prompt injection" among its known model risks by name.

fara-cli is a direct integration. Its install step is playwright install, and it drives a plain local Playwright Chromium — no container, no allow-list, no pause control. The harness README offers the containerised option as an alternative to it rather than as part of it: "instead of fara-cli you can use Magentic-UI", described in the same sentence as "a sandboxed browser environment with auditable action logging and user prompts at critical points". For the CLI path the vendor's position is a recommendation, not something shipped: "We recommend running it in a sandboxed environment, monitoring its execution, and avoiding sensitive data or high-risk domains."

So on the fara-cli path you are the boundary. Practically: a throwaway OS user or a VM, Playwright's own fresh browser profile (never your daily one), signed out of everything, low-stakes public sites, and eyes on the run rather than leaving it unattended. The command in Running passes --headful for exactly that last reason. If you want the container, the allow-lists, watch-mode and the pause control, use Magentic-UI as the client instead.

ℹ️ Vision is the product, not a bonus. Per the model card, "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree." A Fara install that only serves text is a broken install — the --mmproj step below is mandatory, and Step 5 exists to prove that an image really reached the model before you let it drive anything.

⚠️ This install is off the vendor's supported path, deliberately. The card's Requirements section asks for "A set of GPUs with enough memory for a 27B model in bf16 (A6000, A100, H100, and B200 have been tested). We recommend sharding the model over at least 2 GPUs." Everything below runs a community quantisation Microsoft never published or blessed. Q6_K is the closest to the vendor's precision that any consumer card reaches for this model, and it is still not the vendor's precision — see Results.

⚠️ Runtime status, specific to this silicon. llama.cpp support for this architecture is verifiable statically and this recipe shows that evidence rather than asserting it. There is one public report of this model served by llama-server and consumed by an agent harness (see Troubleshooting). What does not exist anywhere public is a log showing an image decode reaching Fara1.5 under llama-server on any GPU, nor a throughput measurement of this model on any consumer card. What does exist on sm_120 is llama.cpp #25717 — an open, community-filed report of a CUDA illegal memory access on vision requests with an mmproj, against a larger model of the same architecture family — and #23385, an open report of fatal aborts in the quantised matmul kernels whose title names this card. Neither is about Fara1.5, and Troubleshooting is explicit about what transfers and what does not. But on this card they are the two things most likely to end your evening, so Step 3's flags and Step 5's second test both exist because of them.

Requirements

Component	Minimum	This recipe
GPU	32GB VRAM, CUDA compute 12.0 (sm_120) for the configuration below	— not measured; budget below is derived (/contribute)
RAM	16GB, 32GB comfortable — see the note below	—
Storage	24.15 GB (Q6_K + f16 mmproj)	24,151,753,184 B = 22.493 GiB, plus the harness and Chromium
Software	CUDA Toolkit 12.8+, llama.cpp b10087+ (CUDA), Python 3.10+, Playwright	—

Three notes on that table.

min_vram_gb is 32, and this is the first Fara1.5 page in the catalogue where it is. It is a filter floor — the smallest real card capacity on which the install below runs — not a measured peak. Two terms push it over 24. The Q6_K weights plus the projector are 22.493 GiB before a single byte of cache. And the --flash-attn off that Step 3 requires on this architecture makes llama.cpp materialise the attention scores in F32, a tensor that scales with context and adds 1.500 GiB at the 32,768-token window this page leads. Net, the documented configuration accounts for 26.578 GiB — over a 24 GiB card outright — so the floor is the next capacity real hardware ships in, which is 32. The 4B and 9B pages for this same card both set 24, because their installs are smaller; the field describes an install, not a card.

On the RAM row. The weights are mmap'd, so most of that 24.15 GB is file-backed rather than resident in VRAM's shadow. The reason to want 32 GB of system RAM is llama-server's prompt cache: its default ceiling is 8 GiB of system RAM (cache_ram_mib = 8192 in common/common.h), and a CUA sending a distinct screenshot every step is exactly the workload that fills it. On a 16 GB machine cap it with --cache-ram; see Troubleshooting.

And the GPU cell is not a measurement — /check reports unknown with zero benchmark rows, and nothing on this page was run on hardware by us.

What 32 GB actually buys

There is no benchmark for this pair, so every number below is arithmetic from the artifacts' own metadata and from llama.cpp's source, not a measurement. Reproduce it rather than trusting it. All totals are GiB (2^30 bytes), the unit a "32GB" NVIDIA card is really in, so they compare directly against 32.

⚠ The 27B is not a scaled-up 9B, and three constants change. If you have read the 4B or 9B pages for this card, do not carry their arithmetic over — including their kq table, which is wrong here. The source config.json declares 64 layers, a layer_types array of 48 linear_attention + 16 full_attention, num_attention_heads: 24, num_key_value_heads: 4, head_dim: 256 and max_position_embeddings: 262144. Against the smaller sizes' 8 attention layers and 16 heads, that doubles the per-token KV cost, triples the recurrent state, and raises the flash-attention-off scores bound by half. The GGUF agrees with the config key for key: the file's own header gives qwen35.block_count 64, attention.head_count 24, attention.head_count_kv 4, key_length 256, value_length 256, full_attention_interval 4, ssm.inner_size 6144, ssm.group_count 16, ssm.state_size 128, ssm.conv_kernel 4.

Weights. Byte counts from the bartowski GGUF tree, quantised on release b10087 with an imatrix:

File	Bytes	GiB
`Fara1.5-27B-IQ4_XS.gguf`	15,328,839,488	14.276
`Fara1.5-27B-Q4_K_S.gguf`	16,474,163,008	15.343
`Fara1.5-27B-Q4_K_M.gguf`	17,533,552,448	16.329
`Fara1.5-27B-Q5_K_M.gguf`	20,513,802,048	19.105
`Fara1.5-27B-Q6_K.gguf`	23,224,145,728	21.629
`Fara1.5-27B-Q8_0.gguf`	28,665,067,328	26.696
`Fara1.5-27B-bf16` (2 shards)	53,808,281,344	50.113
`mmproj-Fara1.5-27B-f16.gguf`	927,607,456	0.864
`mmproj-Fara1.5-27B-bf16.gguf`	931,146,400	0.867

(HuggingFace displays these as decimal GB — 23.22 GB for the Q6_K file — which is the same bytes, not a different number.)

The bf16 row matters more on this card than anywhere else, so read it before the tables. It exists: the quantiser's card labels it "Full BF16 weights." and marks it split across two files. At 50.113 GiB it is 18.113 GiB past even this card. That is not merely a fit question — the 4B and 9B recipes for this same board lead bf16, and leading an unquantised weight type is what let them route around one of the two open Blackwell bugs below. At 27B that escape is closed, and Troubleshooting says exactly what replaces it.

KV cache — 64 KiB per token, twice the smaller sizes. qwen35 is a hybrid architecture: llm_arch_is_hybrid() in src/llama-arch.cpp returns true for LLM_ARCH_QWEN35, and src/models/qwen35.cpp marks layers recurrent off full_attention_interval. Only the 16 full-attention layers carry a cache that grows with context:

K+V per token per attention layer = 2 x 4 x 256 x 2 B (f16) = 4,096 B
x 16 full-attention layers                                  = 64 KiB / token

-c  32,768 ->  2.000 GiB      -c 131,072 ->  8.000 GiB
-c  65,536 ->  4.000 GiB      -c 262,144 -> 16.000 GiB

Recurrent state — fixed, and allocated four times over. The other 48 layers hold a context-independent state, sized by n_embd_r() / n_embd_s() in src/llama-hparams.cpp from the artifact's own GGUF keys. qwen35 sets no n_embd_head_kda, so both fall through to the Mamba forms:

conv state : (4-1) x (6144 + 2 x 16 x 128) =  30,720 elems
ssm  state :         128 x 6144            = 786,432 elems
per layer  : 817,152 x 4 B (F32)           = 3.117 MiB
x 48 linear-attention layers               = 149.625 MiB   per sequence
x 4 sequences                              = 598.5 MiB = 0.584 GiB

That last line is the one most budgets for this model get wrong, and it is worth following because it also explains why the KV cache is not multiplied the same way — and because the same default is what Step 3's --parallel warning turns on. common/arg.cpp sets params.n_parallel = -1 for the server example specifically; tools/server/server.cpp resolves that sentinel with params.n_parallel = 4; params.kv_unified = true;; common/common.cpp then assigns cparams.n_seq_max = params.n_parallel. src/llama-model.cpp sizes the recurrent memory with /* recurrent_rs_size */ std::max((uint32_t) 1, cparams.n_seq_max) on the branch that names LLM_ARCH_QWEN35 explicitly, and src/llama-memory-recurrent.cpp allocates n_rows = mem_size * (1 + n_rs_seq), where n_rs_seq defaults to 0. So the recurrent state is allocated once per slot — four times. The same default line sets kv_unified = true, which gives all four slots one shared attention cache, so the 64 KiB/token figure above is not multiplied.

The accountable ladder. Text weights + f16 projector + KV + the 0.584 GiB recurrent state, in GiB. Each cell sums the exact byte-derived inputs and is rounded once at the end.

Context	Q4_K_M	Q5_K_M	Q6_K	Q8_0
32,768	19.778	22.553	25.078	30.145
65,536	21.778	24.553	27.078	32.145
131,072	25.778	28.553	31.078	36.145
262,144 (native)	33.778	36.553	39.078	44.145

The term this card cannot leave out of the table

On every 24 GB card in this catalogue the ladder above is the whole budget, because flash attention stays on and compute buffers are small and un-derived. Here one of them is neither, and it is a direct consequence of the --flash-attn off in Step 3.

Flash attention exists partly so that the matrix of attention scores never has to exist in memory. Turn it off and llama.cpp takes the explicit branch in src/llama-graph.cpp, which builds kq = ggml_mul_mat(ctx0, k, q) and immediately pins its precision with ggml_mul_mat_set_prec(kq, GGML_PREC_F32), the source's own reason alongside it — "this op tends to require high floating point range". So the graph allocator reserves, for one layer at a time, four bytes per score:

n_kv (= -c)  x  n_ubatch (512 default)  x  n_head (24)  x  4 B (F32)
32,768       x  512                     x  24           x  4  = 1.500 GiB

n_ubatch is 512 by default (common/common.h), and n_head is this model's num_attention_heads: 24 — not the 16 the 4B and 9B carry, which is why their published kq table does not transfer. The term is linear in -c and linear in -ub:

Context	Accountable (Q6_K)	F32 scores, `-ub 512`	Real footprint	Left of 32 GiB
32,768	25.078	1.500	26.578	5.4
65,536	27.078	3.000	30.078	1.9
131,072	31.078	6.000	37.078	— does not fit
262,144 (native)	39.078	12.000	51.078	— does not fit

The model's native 262,144-token window is out of reach here — but the reason is a judgement this page makes, not a wall the card puts up, and it is worth separating the two.

The arithmetic first. At that setting the KV cache alone is 16.000 GiB, which with the projector and the recurrent state gives a 17.448 GiB floor before a single weight byte. Of the tiers in the table above the cheapest, Q4_K_M, totals 33.778 GiB and is out on capacity outright. But bartowski's tree does not stop at Q4: it ships nine tiers below it, from IQ2_M (9.904 GiB) to Q3_K_L (14.008 GiB), and every one of those is accountable inside 32 GiB at the full window — 27.352 to 31.456 GiB, with IQ4_XS at 31.724 also inside. This page's mandatory flash-attention-off surcharge then takes most of them back out: at the default -ub 512 the scores term is 12.000 GiB and nothing fits, and only at -ub 128 (3.000 GiB) do the three cheapest clear the card — IQ2_M at 30.352, Q2_K at 31.252, IQ3_XXS at 31.985.

So a 262,144-token window is reachable on this card, at 2-bit or low 3-bit weights and a quarter-size micro-batch. This recipe declines that, and the reason is editorial rather than physical. Fara's entire output is pixel coordinates; no published evaluation measures what any quantisation tier costs that faculty, and this sweep does not lead sub-Q4 weights for a grounding model on that basis. Spending two or three quant tiers to buy a window whose only consumer is the model's own reasoning — while slowing every image encode with -ub 128 — inverts the trade this page argues for everywhere else. If you weigh it differently, the arithmetic above is the whole of it and the substitution is one word in Step 2. Otherwise cap -c: the vendor's own vLLM Quickstart passes --max-model-len 262144, and that line does not transfer to a consumer board.

The decision: Q6_K at 32,768

Q6_K @ 32,768 with flash attention off = 26.578 GiB, leaving 5.4 GiB. That is the lead, and the reasoning is the sweep's standing one: quantisation cost on this model is unmeasured while context need is bounded and stated, so spend the surplus on the tier rather than the window.

Quantisation cost is unmeasured. Fara's output is pixel coordinates — the card describes its grounding data as "Curated datasets for predicting actions and pixel coordinates from screenshots, with images, text, and bounding boxes." No public evaluation says what any tier costs that faculty. The quantiser's card carries only generic descriptors — Q6_K is "Very high quality, near perfect" against Q4_K_M's "Good quality, default size for most use cases" — which is a size ranking, not a grounding evaluation. It is still the best signal available, and it is why this card's 8 GB of headroom goes here.
Context need is bounded and stated. The harness README asks that "context length is set to at least 15000 tokens and temperature to 0 for best results", and the image half is capped outright — the card says "We only keep the most recent 3 screenshots", which at the derived 1,260 tokens each pins images at 3,780 tokens for a whole run.

Two configurations declined explicitly, so you can disagree with the arithmetic in front of you:

Q8_0 @ 32,768 (30.145 + 1.500 = 31.645 GiB) — not viable. It technically clears 32 GiB by 0.355 GiB, which is less than the CUDA context alone. Do not try it.
Q6_K @ 65,536 at the default -ub 512 (30.078 GiB, 1.9 left) — declined for the same reason. But it is reachable by a different route, below.

The upgrade that costs exactly 2.000 GiB

If you intend a long unattended run, the window is worth buying — and on this card there is a clean way to buy it, because the scores term responds to -ub as strongly as it responds to -c. Halving the physical batch while doubling the window leaves the scores term unchanged:

Configuration	Accountable	F32 scores	Real footprint	Left of 32 GiB
`-c 32768 -ub 512` (lead)	25.078	1.500	26.578	5.4
`-c 65536 -ub 256`	27.078	1.500	28.578	3.4

So the whole cost of doubling the context is the KV doubling — exactly 2.000 GiB — with no scores penalty at all. What -ub 256 costs instead is prompt-processing throughput, which for a CUA means a slower image encode once per step rather than slower generation. That is a good trade for a hundred-round run and a poor one for interactive work you are watching.

⚠ -ub 256 is a memory lever, not a crash mitigation. It appears here because it halves the scores buffer. Do not read it as protection against #25717: that report's only long-running survivor used larger batches (-b 8192 -ub 2048), so batch size is not the axis the crash lives on.

How much context you actually get, per round

fara-cli defaults to --max_rounds 100 — argparse sets default=100 in run_fara.py, and both agent entry points carry the same default in their signatures. Images are capped at three; the text trajectory is not capped at all, and it accumulates for the whole run. So the window divides by rounds:

 32,768 - 3,780 image tokens =  28,988 for text / 100 rounds ~   290 tokens per round
 65,536 - 3,780             =  61,756 for text / 100 rounds ~   618 tokens per round

290 tokens per round is a real budget for a model that emits a chain-of-thought block before every action, but it is a budget, not a guarantee — and context shift is off by default (ctx_shift = false in common/common.h), so an overrun is a hard error rather than a silent truncation of the oldest turns. The rule on this card: lead -c 32768 for interactive work; move to -c 65536 -ub 256 before a long unattended run or if you watch a run and see long <think> traces. Do not raise --max_rounds past 100 without raising -c with it.

Nothing in the tables above is free space you can plan against to the last byte: the CUDA context, the remaining compute buffers and the vision encoder's working set are allocated at load time and this recipe does not derive them. Read the CUDA0 compute buffer size lines llama-server prints on startup, watch nvidia-smi with the agent running, and report what you see.

How the context is actually spent

A CUA sends a screenshot every step, so the per-image cost decides whether the budget above is realistic. It is not a fixed number baked into the projector: llama.cpp treats this one as dynamic-resolution, which is what --image-min-tokens exists to floor.

The harness fixes the browser viewport at 1440x900 (viewport_width: int = 1440, viewport_height: int = 900 in fara15_agent.py), matching the card's note that "The screen resolution Fara is most commonly trained with is" 1440x900. It pre-resizes with Qwen's smart_resize at patch size 16 and merge size 2, so the factor is 32 and 900 becomes 896. llama.cpp then counts merged patches per axis in tools/mtmd/clip.cpp, where the PROJECTOR_TYPE_QWEN3VL branch returns (img->nx() / params.patch_size) / 2 across and the same down:

across : (1440 / 16) / 2 = 90 / 2 = 45
down   : ( 896 / 16) / 2 = 56 / 2 = 28
tokens : 45 x 28                 = 1,260 per screenshot
x max_n_images = 3               = 3,780 tokens resident

The source config.json's vision_config agrees on the geometry (patch_size: 16, spatial_merge_size: 2, depth: 27, out_hidden_size: 5120), and so does the projector artifact's own header (clip.vision.patch_size 16, spatial_merge_size 2, block_count 27). The harness's conservative default is image_token_estimate: int = 1500, a useful sanity check on the arithmetic.

One consequence matters for the crash risk on this silicon, and it is easy to miss: because the image history is capped but the text history is not, the number of text tokens sitting in front of each screenshot grows with every round. That is one of the variables #25717 identifies as a trigger. Step 5's second test probes it deliberately.

Installation

1. Build llama.cpp with CUDA 12.8+ for sm_120

This is the one step that genuinely differs on Blackwell. The RTX 5090 is compute capability 12.0, and llama.cpp's CUDA CMakeLists annotates that target as "120 == Blackwell, needs CUDA v12.8, FP4 tensor cores". Check the toolkit first, because this is the only reliable way to check it:

nvcc --version    # must report release 12.8 or newer

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

Two details worth knowing rather than pasting:

Pass 120 explicitly — but not because an older toolkit would silently skip your card. It would not: the default path never reaches the architecture list at all. The same file takes set(CMAKE_CUDA_ARCHITECTURES "native") whenever GGML_NATIVE is on, which it is by default, so on an ordinary build an inadequate toolkit fails loudly. Two real reasons to name the architecture anyway: it makes the build reproducible rather than dependent on which GPU CMake happened to see, and it closes the native path's one soft failure — docs/build.md documents nvcc warning : Cannot find valid GPU for '-arch=native', default arch is used, a warning you can scroll past on the way to a binary that does not target your card.
You write 120, the build system compiles 120a. The same file rewrites any plain 12X architecture into its architecture-specific 12Xa form, in its own comment because the Blackwell instructions "are not forwards compatible". You do not type the a, and you should not be surprised to see 120a-real in the CMake status line. Do not read that line as a toolkit check — Troubleshooting explains why it proves nothing.

Build from a release at or after b10087 — the release the GGUF used here was quantised on. That is also newer than the builds #25717 was filed against, which is one reason to test rather than assume the report reproduces.

Both halves of the model are first-class in mainline — worth confirming before you download twenty-four gigabytes:

grep -n 'LLM_ARCH_QWEN35,' src/llama-arch.cpp        # -> { LLM_ARCH_QWEN35, "qwen35" }
grep -n 'qwen3vl_merger' tools/mtmd/clip-impl.h      # -> PROJECTOR_TYPE_QWEN3VL

A note on FP4, since Blackwell is why people ask. The FP4 tensor cores are real and llama.cpp does use them: GGML_TYPE_NVFP4 is a shipped quant type in ggml/include/ggml.h with a native Blackwell path in mmq.cu. What is missing is not the loader — it is an artifact: no quantiser has published an NVFP4 build of Fara1.5-27B. And FP4 is a sub-4-bit tier, i.e. the opposite direction from what this card's 32 GB is for. Build for 120 because that is your architecture, not because it unlocks anything here.

2. Download the weights and the projector

Both files are required. The projector is a separate artifact — downloading only the text GGUF gives you a model that cannot see.

pip install -U huggingface_hub

hf download bartowski/Fara1.5-27B-GGUF \
  Fara1.5-27B-Q6_K.gguf mmproj-Fara1.5-27B-f16.gguf \
  --local-dir ~/models/fara1.5-27b

That is 24.15 GB over the wire, so start it before you read the rest of the page.

Microsoft publishes no first-party GGUF for Fara1.5 — the microsoft org ships Fara1.5-4B, Fara1.5-9B and Fara1.5-27B as safetensors only, plus an ONNX build of the previous-generation Fara-7B, which is a different model. Four independent community quantisers have converted the 27B including its vision tower — bartowski, prithivMLmods, Abiray and DevQuasar each publish a matching mmproj. Four independent conversions succeeding is the strongest available evidence that the vision path converts cleanly; it is still not a runtime test. Their quant recipes differ, so the budget above applies only to the file it names.

On the two projector files. bartowski publishes the projector twice, -f16 and -bf16, and one of the two preserves the source bytes — but config.json is the wrong place to work out which. Reading the safetensors headers directly, every shipped tensor is BF16, vision tower included; the 27B's config.json in fact declares "dtype": "float32" under both text_config and vision_config while its top-level dtype says bfloat16, and the weights side with the top level. So mmproj-Fara1.5-27B-bf16.gguf is the byte-faithful conversion here and mmproj-…-f16.gguf is a re-encode — one that keeps every bit of bfloat16's 7-bit mantissa (binary16 has 10) and gives up bfloat16's wider exponent range. This recipe still takes the f16 build so the page stays comparable with every other Fara1.5 recipe in the catalogue; the bf16 build is a one-word substitution costing 3,538,944 B — 3.375 MiB, moving the accountable total from 25.078 to 25.081. Nobody has measured which grounds better.

3. Serve it

./build/bin/llama-server \
  -m ~/models/fara1.5-27b/Fara1.5-27B-Q6_K.gguf \
  --mmproj ~/models/fara1.5-27b/mmproj-Fara1.5-27B-f16.gguf \
  -ngl 99 -c 32768 --flash-attn off \
  --host 127.0.0.1 --port 5000 \
  --image-min-tokens 1024 \
  --reasoning-format none \
  --temp 0

Four of those flags are load-bearing and none is a default:

--flash-attn off — the one that is specific to this silicon. common/arg.cpp documents the flag as [on|off|auto] with the struct default auto, and src/llama-context.cpp sets cparams.flash_attn = params.flash_attn_type != LLAMA_FLASH_ATTN_TYPE_DISABLED, so auto starts out enabled and is then confirmed by a runtime probe. Flash attention on is the strongest trigger in #25717's crash matrix — the only variable that crashed on its own, with a short prompt. Passing nothing here would document the crashing configuration. It is not protection, and it is not free. Two rows of that matrix crash with it off, so read it as removing a necessary condition rather than as a fix; and see the F32 scores term above, which is what it costs and what sets this page's floor.
--reasoning-format none — mandatory. Without it the agent loop silently receives nothing; Troubleshooting has the mechanism. arg.cpp documents this value as "none: leaves thoughts unparsed in message.content".
--image-min-tokens 1024 — this projector ships no minimum-pixel key, so llama.cpp falls back to the floor set by hparams.set_limit_image_tokens(8, 4096) in the PROJECTOR_TYPE_QWEN3VL branch of tools/mtmd/clip.cpp, and warns a few lines later in the same branch: "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks", pointing at issue #16842 on bounding-box correctness. Fara is a grounding model. At the harness's viewport the image is already 1,260 tokens and the floor never binds; it protects you if you shrink the viewport.
-c 32768 — see the budget tables. Not the model's native maximum, which does not fit even this card.

And one flag to deliberately not pass: --parallel. This is the other half of the #25717 mitigation, and it is a subtraction rather than an addition.

The two configurations in that crash matrix which completed both ran a unified KV cache — and on llama-server that is already the default, so you get it by leaving the flag alone: the n_parallel = -1 sentinel resolved by server.cpp into n_parallel = 4; kv_unified = true; is the same line the recurrent-state derivation above hangs on. Passing --kv-unified yourself is therefore a no-op here, which is why it is absent from the command above; it would also be the wrong lesson to draw, since one of the crashing rows passes it explicitly.

What matters is the opposite move. Passing any explicit --parallel N takes n_parallel out of the auto sentinel, so the server block never runs, kv_unified stays at its false struct default, and src/llama-context.cpp then computes n_ctx_seq = n_ctx / n_seq_max — splitting the cache, which is one of the report's three triggers. Both flash-attention-off crashes in the matrix passed --parallel. On this page's window it would also take each slot to 8,192 tokens, below the harness's documented 15,000-token floor — a second, independent reason.

4. Confirm what actually loaded

./build/bin/llama-server --version    # build number >= 10087

Four lines of the startup log are worth reading before going further: the CUDA0 device line (it should name your 5090 — if the model loaded on CPU, go back to Step 1), the flash_attn line (it must say disabled; if it does not, your flag did not take), the clip_ctx: CLIP using <backend> backend line — clip.cpp emits it as "CLIP using %s backend" with the backend's own name, and CPU there means the projector is not on the GPU — and the mmproj load stage itself.

5. Prove the vision path works — twice, and the second time is the one that matters

Check that the server advertises multimodal support, then send a real image. The server docs make the first part a client-side requirement: "Clients should check /models or /v1/models for the multimodal capability before a multimodal request." Run GET /props alongside it — it reports which modalities loaded rather than merely that some did.

curl -s http://127.0.0.1:5000/v1/models | grep -o multimodal

# supplementary — names the modality that actually loaded
curl -s http://127.0.0.1:5000/props | python3 -c "import sys,json; print(json.load(sys.stdin)['modalities'])"
# expect: {'vision': True}

# test A — short prompt, image first
curl -s http://127.0.0.1:5000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "fara",
  "temperature": 0,
  "messages": [{"role": "user", "content": [
    {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}},
    {"type": "text", "text": "Describe this image in one sentence."}
  ]}]
}' | python3 -m json.tool

Test A is not sufficient on this silicon. In #25717's matrix a short prompt with flash attention off completed, and longer-prompt runs with flash attention off did not — those rows also passed --parallel, which the command above deliberately omits, so prompt length was not the sole difference; but a real agent prompt is the long one either way. So run the second test before you trust the setup:

# test B — ~800 tokens of text BEFORE the image, which is the agent-run shape
python3 - <<'PY'
import json, urllib.request
long_text = "Describe precisely. " * 200
body = {"model": "fara", "temperature": 0, "max_tokens": 200,
        "messages": [{"role": "user", "content": [
            {"type": "text", "text": long_text},
            {"type": "image_url", "image_url": {"url":
             "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}]}]}
req = urllib.request.Request("http://127.0.0.1:5000/v1/chat/completions",
        data=json.dumps(body).encode(), headers={"Content-Type": "application/json"})
print(urllib.request.urlopen(req, timeout=600).read()[:400])
PY

If both replies describe the picture, you have cleared the risk this page is most worried about. If either fails silently — the model answering as though no image were present — stop; the agent loop below will produce confidently wrong actions rather than fail loudly. If test B kills the server, watch the log for a find_slot: non-consecutive token position line immediately before the CUDA error and go to Troubleshooting: that would be the reported failure reproduced on a new model, a different Blackwell card and a newer build, which is worth a comment on the upstream issue.

6. Install the harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Skip the README's [vllm] extra — you are hosting with llama-server, and it pulls a large dependency tree you will not use.

Running

Two clients share the same llama-server endpoint. Path A (fara-cli) is the smoke test, with no isolation. Path B (Magentic-UI) is what the vendor recommends for anything real. They are not equivalent in risk; pick deliberately.

Path A — `fara-cli`, for a first smoke test

fara-cli speaks to any OpenAI-compatible endpoint, but its built-in defaults are for a different size: DEFAULT_ENDPOINT_CONFIG in run_fara.py is {"model": "Fara1.5-9B", "base_url": "http://localhost:5000/v1", "api_key": "not-needed"}. The port lines up; the model name does not, so pass it explicitly.

source .venv/bin/activate
fara-cli \
  --base_url http://127.0.0.1:5000/v1 \
  --api_key sk-no-key-required \
  --model fara \
  --task "find the current UTC time on time.is and tell me" \
  --output_folder ./runs/first \
  --save_screenshots \
  --headful \
  --max_rounds 15

--headful is in that command on purpose. The harness runs the browser headless by default — run_fara.py declares the flag with the help text "Run the browser in headful mode (show GUI, default is headless)" and both launch sites read headless=not args.headful. Either way this drives a real, unisolated Chromium on your machine; the danger is identical with or without the flag. What the flag changes is whether you can see it, and the supervision advice above assumes you can.

Each step writes a screenshot and a data_point.json into --output_folder, which is how you tell a grounding failure (clicks landing in the wrong place) from a reasoning failure. Start with the low --max_rounds 15 above for a second reason on this card: if the prompt-cache growth in the report cited in Troubleshooting is real on a source build, it shows up on long runs, not short ones.

Keep the task on a low-stakes public site and stay signed out. Fara is trained to stop and ask before entering personal information, making payments, submitting forms, signing in or sending messages; if you see it sail through one of those, stop the run — that is a wiring problem in your prompt path, not a quirk to tolerate.

Path B — Magentic-UI, for the container

Everything the model card assumes — the containerised browser, domain allow-lists, watch-mode and a pause control — lives in Magentic-UI, not in fara-cli. That is also where a Docker requirement would come from; there is no Docker step in Path A because Path A genuinely does not use one.

Magentic-UI takes the same OpenAI-compatible endpoint you started above, so nothing on this page changes except the client. Its setup is documented in its own repository rather than reproduced here, because this recipe has not verified those steps end to end on this GPU — give it http://127.0.0.1:5000/v1 as the model endpoint. Whichever client you use, match the trained resolution: the card is explicit — "Match this in your sandbox for the most reliable grounding."

Results

Speed: omitted — no published measurement of Fara1.5-27B exists for this card or for any consumer GPU, and the only hardware Microsoft names is datacenter class. Borrowing a figure from an A100 for a 32 GB Blackwell card would be misleading, and the vendor publishes no throughput numbers to borrow in the first place. If you run this, please contribute your numbers so this line can be replaced with a measurement — and if you A/B against Q4_K_M, that pair of step latencies is the most useful thing anyone could add to this page.
VRAM usage: 22.493 GiB of weights and projector; 25.078 GiB accountable including the f16 KV cache and the four-slot recurrent state at a 32,768-token context, plus a bounded 1.500 GiB of F32 attention scores because this card's configuration disables flash attention — 26.578 GiB in total, and the reason min_vram_gb is 32. All of it is derived above from byte counts and llama.cpp's memory formulas, not observed; the CUDA context, the remaining compute buffers, the vision graph and the browser's own GPU memory still sit on top. Track the empirical picture at /check/fara1-5-27b/rtx-5090.
Quality notes. Fara's whole job is emitting precise pixel coordinates, and quantisation of a grounding model is a real risk rather than a formality. No evaluation of how any quant tier affects Fara1.5's grounding accuracy exists — not from Microsoft, not from the community. The vendor's published scores for this size (89.3 WebVoyager, 72.3 Online-Mind2Web, 40.2 WebTailBench, from the model card's evaluation table) are for the bf16 model on datacenter hardware; nothing licenses carrying them to a 6-bit build. What this card buys is distance from that question, not an answer to it: Q6_K is two tiers above what a 24 GB card can hold for this model, and it is still not the precision that was evaluated.
What Blackwell changes, and what it does not. Nothing about this model is CUDA-generation-dependent by design — the architecture is fully registered in mainline llama.cpp and the multimodal layer contains no architecture-specific code. What sm_120 changes is the maturity of the kernels and drivers underneath, which is what the first three Troubleshooting entries are about.
Language: English only — the card puts "Languages other than English (training data is English-only)" out of scope.

For the full benchmark data, see /check/fara1-5-27b/rtx-5090.

Troubleshooting

The server dies on an image request with `CUDA error: an illegal memory access was encountered`

This is the failure to know about on sm_120, and it is why Step 5 has two tests. llama.cpp #25717 reports llama-server crashing with exactly that error on vision requests, preceded in the log by find_slot: non-consecutive token position. The reporter bisected seven configurations; the triggers are flash attention, a split KV cache, and — the surprising one — the amount of text placed before the image in the prompt. Two rows completed: flash attention off with the default unified cache and a short text prefix, and one where the same long prompt was sent with the image as the first content part.

Read it with its limits stated, because they matter and because the mitigation is shaped by them rather than by the headline:

The architecture family is this one and the silicon family is this one, but the card is not. The report's own hardware line is an RTX 5070 Ti 16 GB on sm_120. Nobody has filed the same failure against this card.
It is a community report with no maintainer response. One comment exists and it is the reporter's own follow-up; the filer has no association with the project, and no pull request references it. Its newest build predates the b10087 floor Step 1 pins.
The model is not this one. It was filed against a 35B-A3B mixture-of-experts member of the same family — a different architecture enum, a different layer count and KV-head count — quantised, with expert CPU offload (--n-cpu-moe 24) and a 4096-token physical batch in play. None of that MoE configuration transfers, and a fully-offloaded dense model at the default -ub 512 exercises a materially simpler memory plan.
The implicated mechanism does transfer, and that is checkable rather than assumed. The source config.json sets rope_parameters.mrope_interleaved: true with mrope_section: [11, 11, 10] — this model uses the same M-RoPE positioning the report blames — and it loads the same qwen3vl_merger projector. Separately, the non-consecutive token position line that precedes the crash is emitted from src/llama-memory-recurrent.cpp, i.e. from the recurrent half of the hybrid cache, which 48 of this model's 64 layers traverse — a larger share than in any smaller Fara1.5. That is not MoE-specific.

So: a real, specific risk worth shaping the command around, not a demonstrated failure of this configuration. What this recipe does about it is cheap — --flash-attn off plus not passing --parallel reproduces the layout of the rows that completed, and Step 5's test B probes the long-text-prefix trigger in ten seconds. Do not read --flash-attn off as protection: two rows of the matrix crash with it off. And note that --no-mmproj-offload does not dodge this bug — one crashing row ran the vision encoder on the CPU with flash attention on and crashed anyway. It is a VRAM lever, listed under "out of memory" below for that reason and no other.

One piece of luck is on your side, and it is worth more than the flags: the harness builds every user turn with the screenshot as the first content part and the text after it — content=[ImageObj.from_pil(scaled_screenshot), text_prompt] in fara15_agent.py, at both call sites — which is the ordering of the only long-prompt row that completed. Fara's system prompt is still long text ahead of the first image in the flattened sequence, and that prefix grows with every round, so the shape is not exactly either reported case. If test B crashes and test A does not, you have reproduced the report on a new model, a different card and a newer build — post it.

A fatal abort in the MMQ kernels, or a CUDA invalid-value at kernel launch

llama.cpp #23385 is open and names this card in its own title — "Eval bug: Fatal MMQ crashes on Blackwell (RTX 5090/5080) due to unhandled sharedMemPerBlockOptin driver bug" — with a model line of "Any model". The root cause is a driver defect rather than a llama.cpp one: early Blackwell drivers return a corrupted cudaDeviceProp.sharedMemPerBlockOptin, and that value flows into the shared-memory budget the quantised matmul kernels dispatch against. mmq.cu has since added a floor — its own comment reads "MMQ tiles require at least 48 KiB per-block shared memory; fall back to BLAS otherwise." — which turns the zero case into a cuBLAS fallback rather than an abort; the overflowed variant clears that floor trivially and is not covered by it. The issue is community-filed with three comments and no maintainer fix merged for the MMQ path.

Here is the part that is specific to the 27B, and it is the main reason this section is longer than its 4B and 9B counterparts. Those pages defuse this issue by leading bf16, on a mechanism you can read: ggml_cuda_should_use_mmq in mmq.cu switches on the weight type, enumerating Q4_0/Q4_1/Q5_0/Q5_1/Q8_0, Q2_K through Q6_K, the IQ* family, MXFP4 and NVFP4 as supported, and falling to default: mmq_supported = false for everything else — bf16 and f16 included. A bf16 model therefore never enters the affected path at all.

That escape does not exist at 27B, and not for the reason you might guess. A bf16 GGUF of this model does exist; it is 50.113 GiB across two shards, i.e. 18.113 GiB past this card. So every weight file that fits a 32 GB board — Q6_K included, and Q4_K_M, Q5_K_M, IQ4_XS, all of them — is a type ggml_cuda_should_use_mmq accepts. The escape is closed by capacity, not by availability, and no choice of quant reopens it.

What replaces it is a build flag rather than a weight choice, and it is in the same function: ggml_cuda_should_use_mmq opens with #ifdef GGML_CUDA_FORCE_CUBLAS return false; before it looks at the type at all. ggml/CMakeLists.txt exposes that as a real option, described as "ggml: always use cuBLAS instead of mmq kernels". So, in order:

Update the graphics driver. The root cause is the driver, and this is the only fix that costs nothing.
Rebuild with -DGGML_CUDA_FORCE_CUBLAS=ON. This routes every weight type around MMQ, so unlike the smaller Fara1.5 recipes' answer it does not depend on which quant you loaded. It costs throughput — the MMQ kernels exist because they are faster than cuBLAS on quantised weights — so treat it as a diagnosis and a fallback rather than a default.
There is no third option here. Dropping a quant tier does not help: every tier on this ladder is MMQ-eligible.

The build succeeds but the GPU is idle, or the server falls back to CPU

Check the toolkit directly first: nvcc --version must report 12.8 or newer. Blackwell's instructions are not forwards-compatible, so an older toolkit cannot emit sm_120a code at all.

Be aware, though, that a too-old toolkit is the least likely explanation for an idle GPU, because it normally fails the build outright rather than producing a working binary with no kernels for your card (Step 1 has the mechanism). The two explanations that actually reach a running server are a -arch=native build where CMake could not see the GPU — grep the build log for the Cannot find valid GPU warning docs/build.md documents — and a binary you did not compile yourself.

Do not try to confirm the toolkit from the configure log. The 12X → 12Xa rewrite in the CUDA CMakeLists is unconditional — it sits outside the if (NOT DEFINED CMAKE_CUDA_ARCHITECTURES) block that holds the toolkit-version gate, and Step 1 passes -DCMAKE_CUDA_ARCHITECTURES=120 explicitly, which skips that block entirely. So Replacing 120 … with 120a prints on every toolkit version and proves nothing. nvcc --version is the test.

Relatedly: build from source rather than reaching for a third-party "sm120 Blackwell" binary. #26205 is an open report of llama-server crashing silently before it starts listening on an sm_120 card under exactly that kind of prebuilt Windows build, with an official older-CUDA build giving 0% GPU utilisation on the same machine — the two failure modes this step exists to prevent, in one report. Its hardware line names a different Blackwell card, so read it as an argument for building rather than as a report about this one.

The agent does nothing — every step logs an empty response or `list index out of range`

This is the failure everyone hits, and it is not a bug in your setup. Fara opens a <think> block and never closes it. Under llama-server's default reasoning extraction the parser therefore treats the whole output — reasoning and the <tool_call> block together — as reasoning, returns it in reasoning_content, and leaves message.content empty. The harness parses message.content for <tool_call>, finds nothing, and the loop dies on round one.

Launch with --reasoning-format none. The raw output then begins with a stray unclosed <think> tag, which is cosmetic.

This was reported against this exact model — a community user on llama.cpp b10107 with the 27B, in the model's HF discussions, who documents the repro as "default reasoning format → content: \"\", thoughts + <tool_call> block in the reasoning field, finish_reason: stop" against "--reasoning-format none → full raw output in content, parses fine", and adds "If you have a router/proxy in front doing its own reasoning extraction, disable it there too." Microsoft acknowledged it there — swhitehead, whose HF account carries the org-member flag on that repo, replied "Would you mind posting this as an issue on the Fara1.5 github repo so we can mention this in the hosting instructions?" — which became microsoft/fara#82, still open at the time of writing. PR #83 proposes reading reasoning_content as a harness-side fallback; until it lands, the server flag is the fix.

The model answers but ignores the screenshot

Almost always the projector. Confirm --mmproj is on the command line, that the server logged an mmproj load stage and a CLIP using … line, that /v1/models advertises the multimodal capability, and that Step 5's /props call reports vision: True. A text-only GGUF with no projector loads and runs perfectly happily — it simply cannot see, which in a CUA presents as confidently wrong actions rather than an error.

Clicks land in the wrong place

Check the viewport first: the harness defaults to 1440x900 because that is what the model was mostly trained at, and a different window size changes both the grounding distribution and the token cost derived above. Then confirm --image-min-tokens 1024 is set, since llama.cpp's own warning ties sub-1024-token images to degraded grounding. Only after both should you suspect the weights — and on this card the one rung above Q6_K that exists (Q8_0, at 31.645 GiB with the scores term) does not leave enough of the board to be worth trying. If a task genuinely depends on coordinate precision, the honest test is a bf16-hosted endpoint elsewhere.

A long agent run climbs in system RAM, or the server dies after many steps

llama-server's prompt cache and context checkpoints are sized for chat, not for a stream of unique screenshots: the defaults are cache_ram_mib = 8192 and n_ctx_checkpoints = 32 per common/common.h. #26216 reports the cache growing past its own cap and a deterministic segfault under sustained unique-prompt vision load — the exact shape of a long CUA run. Three caveats before you treat it as a property of your build: it was closed as not_planned and drew no discussion, it was reproduced on the project's container images rather than a source build, and its host line names a different Blackwell card. The knobs are cheap regardless: cap the cache with --cache-ram 2048 (or 0 to disable it) and reduce the checkpoint count if a many-round run climbs. This is a system-RAM problem, not a VRAM one — 32 GB of VRAM does not help.

Out of memory at load

26.578 GiB of real footprint against 32 GiB gives you room, but the levers here are not the ones the 24 GB recipes reach for, because the F32 scores term responds to a knob they never touch.

Cheapest first: halve -ub. The scores tensor is n_kv × n_ubatch × n_head × 4 B, so it is linear in the physical batch size as well as in the context. -ub 256 takes it from 1.500 GiB to 0.750 GiB, and -ub 128 to 0.375 GiB, at the cost of slower prompt processing — which for a CUA means a slower image encode once per step, not slower generation. Unlike halving the window it costs you nothing in trajectory length, which is why it goes first.

Then -c: 32768 → 16384 frees 1.000 GiB of KV and 0.750 GiB of scores, 1.750 GiB in total, and still clears the harness's 15,000-token floor.

Then the quant ladder: Q6_K → Q5_K_M frees 2.524 GiB and Q6_K → Q4_K_M frees 5.300 GiB, which is what every 24 GB card in this catalogue runs. Both are steps down an unmeasured grounding ladder, so take them after the two levers above.

Do not reach for the usual --cache-type-k q8_0 --cache-type-v q8_0 here — the V half is incompatible with the --flash-attn off this card's configuration needs. src/llama-context.cpp rejects that combination outright, logging "quantized V cache requires flash_attn to be enabled" and returning a null context rather than starting. Quantising the K half alone is legal and needs no flash attention: block_q8_0 is one ggml_half scale plus 32 int8_t quants, 34 bytes per 32 values against f16's 64, so the cache drops from 64 to 49 KiB/token — 1.531 GiB instead of 2.000 GiB at a 32,768-token context. And --no-mmproj-offload keeps the projector in system RAM for 0.864 GiB, at the cost of slower image encoding once per agent turn — but see the #25717 entry above before you assume it also helps with the crash, because it does not.

Don't reach for speculative decoding to fill the headroom

The model family carries a multi-token-prediction layer — the source config.json declares mtp_num_hidden_layers: 1 inside text_config. That layer is not part of the GGUF conversion path used here: llama.cpp names the corresponding header key nextn_predict_layers (LLM_KV_NEXTN_PREDICT_LAYERS in src/llama-arch.cpp), and the file this recipe downloads carries no blk.*.nextn.* tensors for --spec-type to attach to. Nothing on this page enables it; if you go looking, that is why it does nothing.

`rope.dimension_sections has wrong array length; expected 4, got 3`

This is a real open llama.cpp bug (PR #25334, titled "fix: load qwen35 Ollama GGUF exports"), but it belongs to Ollama-exported qwen35 blobs, not to the GGUF this recipe uses. Reading this file's own header, qwen35.rope.dimension_sections is [11, 11, 10, 0] — four entries, the trailing zero llama.cpp expects. If you see this error, you are loading a different file than the one downloaded in Step 2. Relatedly, ollama run fara does nothing: there is no Ollama library entry for Fara1.5 in any form. The GGUF / LM Studio / Ollama line in the harness README refers to Fara-7B, the previous generation, reached with a --fara-7b flag.

Generation is slower than you expected

One lead worth knowing about rather than a known property: llama.cpp issue #25162 is an open, community-reported regression against exactly the SSM/linear-attention kernels this architecture leans on — 48 of this model's 64 layers — titled "Performance regression on Turing GPUs after 9e58d4d69 — Qwen35 SSM kernels affected". Read its scope carefully: the title says Turing, the reporter measured on an RTX 2080 Ti, its two comments are both community accounts with no maintainer response, and nobody has posted numbers for a Blackwell card. It is a thing to check with llama-bench across a couple of builds, not a property of this one. Separately, if you built with -DGGML_CUDA_FORCE_CUBLAS=ON from the MMQ entry above, that alone will cost you prompt-processing throughput on quantised weights — turn it back off once the driver is updated.