Fara1.5-9B on RTX 3090: BF16 Computer-Use Agent with Vision under llama.cpp

What You'll Build

A local browser computer-use agent (CUA): Microsoft's Fara1.5-9B served by llama-server on an RTX 3090, with the vision projector loaded so the model actually sees browser screenshots, driven by Microsoft's own fara-cli harness against a plain local Playwright Chromium.

This is the first card in the catalogue where the 9B's unquantised BF16 weights are reachable at all. Every smaller card has to answer a question nobody has measured: how much grounding accuracy does quantisation cost a model whose output is pixel coordinates? On 24 GB you decline the question, and run the same precision Microsoft evaluated.

Hardware data: RTX 3090 (24GB VRAM) · BF16 weights + f16 mmproj = 17.545 GiB · 19.741 GiB accountable at a 65,536-token context (derived below) · See benchmark data

⚠️ This model clicks, types and submits in a real browser, and the path documented here gives it no isolation of any kind. That is the operating requirement, not boilerplate, and it is the thing to settle before anything else on this page.

The model card is blunt about the consequence: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." That sentence closes the card's Recommended Deployment: MagenticLite section, which lists four controls as MagenticLite features — the first being "the browser runs in a Docker container with no access to host files or environment variables" — and then says: "If you integrate Fara1.5-9B directly, you're responsible for these controls." The card's Responsible AI section asks you to "run Fara in an isolated container with no access to host files, environment variables, or sensitive credentials". Fara also reads whatever is on the page, so a hostile page is a prompt-injection surface — the card lists "Prompt injection" among its known model risks by name.

fara-cli is a direct integration. Its install step is playwright install, and it drives a plain local Playwright Chromium — no container, no allow-list, no pause control. The harness README offers the containerised option as an alternative to it rather than as part of it: "instead of fara-cli you can use Magentic-UI", described in the same sentence as "a sandboxed browser environment with auditable action logging and user prompts at critical points". For the CLI path the vendor's position is a recommendation, not something shipped: "We recommend running it in a sandboxed environment, monitoring its execution, and avoiding sensitive data or high-risk domains."

So on the fara-cli path you are the boundary. Practically: a throwaway OS user or a VM, Playwright's own fresh browser profile (never your daily one), signed out of everything, low-stakes public sites, and eyes on the run rather than leaving it unattended. The command in Running passes --headful for exactly that last reason. If you want the container, the allow-lists, watch-mode and the pause control, use Magentic-UI as the client instead.

ℹ️ Vision is the product, not a bonus. Per the model card, "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree." A Fara install that only serves text is a broken install — the --mmproj step below is mandatory, and Step 4 exists to prove that an image really reached the model before you let it drive anything.

⚠️ Runtime status — read before you commit an evening to this. llama.cpp support for this architecture is verifiable statically, and this recipe shows that evidence rather than asserting it. What does not exist anywhere public is a report of Fara1.5 decoding an image under llama-server or llama-mtmd-cli, on this card or on any other. Nor is there a throughput measurement for this model on any consumer GPU — see Results for why there is no speed figure here. Treat your first run as the test, and use Step 4, which reads llama.cpp's own load-time self-report rather than asking you to trust this page.

Requirements

Component	Minimum	This recipe
GPU	24GB VRAM, CUDA compute 8.0+ (Ampere or newer, for BF16 tensor cores)	— not measured; budget below is derived (/contribute)
RAM	16GB (weights are mmap'd on their way to VRAM; Playwright's Chromium sits on top)	—
Storage	18.84 GB (BF16 + f16 mmproj)	18,838,863,264 B = 17.545 GiB, plus the harness and Chromium
Software	llama.cpp b10087+ (CUDA), Python 3.10+, Playwright	—

Two notes on that table. min_vram_gb is 24, and unlike the 4B recipes for this card that is not a judgement call. It is the filter floor for the install documented here — the smallest real card capacity this configuration runs on — and the BF16 text weights are 16.690 GiB on their own, which is 0.690 GiB over a 16 GiB card before a projector or a single byte of cache. The 16 GB tier is excluded by arithmetic, not by margin, and no context setting rescues it. The next real capacity above 16 is 24, so that is the floor. And the GPU cell is not a measurement — /check reports unknown with zero benchmark rows, and nothing on this page was run on hardware by us.

Microsoft ships safetensors only, all of it in BF16: the repo's own config.json declares "dtype": "bfloat16" for the text model. Its Quickstart serves those with vLLM (--dtype bfloat16) and names datacenter hardware for the job: "A GPU with enough memory for a 9B model in bf16 (A6000, A100, H100, and B200 have been tested)", and the card's highlights claim only that "9B parameters means it runs on a single A100/H100/B200 with room for the screenshot history". What this recipe shows is that the precision in those sentences is available on a 24 GB consumer card.

What 24 GB actually buys

There is no benchmark for this pair, so every number below is arithmetic from the artifacts' own metadata and llama.cpp's source, not a measurement. Reproduce it rather than trusting it. All totals are GiB (2^30 bytes), the unit a "24GB" NVIDIA card is really in, so they compare directly against 24.

Weights. Byte counts from the bartowski GGUF tree:

Component	Bytes	GiB
Fara1.5-9B-bf16.gguf	17,920,697,216	16.690
mmproj-Fara1.5-9B-f16.gguf	918,166,048	0.855
Weights total	18,838,863,264	17.545

KV cache — and this is where Fara1.5 is unusual. Its qwen35 architecture is a hybrid: llm_arch_is_hybrid() in src/llama-arch.cpp returns true for LLM_ARCH_QWEN35, and src/models/qwen35.cpp marks layers recurrent off the full_attention_interval. The source config.json spells the split out layer by layer: its layer_types array holds 24 linear_attention entries and only 8 full_attention, against num_hidden_layers: 32 and full_attention_interval: 4. Only those 8 carry a cache that grows with context.

With num_key_value_heads = 4 and head_dim = 256:

K+V per token per attention layer = 2 x 4 x 256 x 2 B (f16) = 4,096 B
x 8 full-attention layers                                   = 32 KiB / token

A non-hybrid 32-layer model with the same head geometry would cost four times that. Context on this model is cheap — doubling -c costs exactly 1.000 GiB per doubling from 32,768 — and it is cheap at 9B for precisely the same reason it is cheap at 4B: this family keeps hidden_size 4096, head_dim 256 and the same 24/8 layer split across sizes, and grows intermediate_size instead. The per-token cache of the 9B is byte-for-byte the 4B's.

Recurrent state — fixed, and allocated four times over. The other 24 layers hold a context-independent state, sized by n_embd_r() / n_embd_s() in src/llama-hparams.cpp from the artifact's own GGUF keys (ssm.conv_kernel 4, ssm.inner_size 4096, ssm.group_count 16, ssm.state_size 128 — the config's linear_conv_kernel_dim, linear_num_value_heads x linear_value_head_dim, linear_num_key_heads and linear_key_head_dim respectively). qwen35 sets no n_embd_head_kda, so both fall through to the Mamba forms:

conv state : (4-1) x (4096 + 2 x 16 x 128) =  24,576 elems
ssm  state :         128 x 4096            = 524,288 elems
per layer  : 548,864 x 4 B (F32)           = 2.094 MiB
x 24 linear-attention layers               = 50.25 MiB   per sequence
x 4 sequences                              = 201 MiB = 0.196 GiB

That last line is the one most budgets for this model get wrong, and it is worth following because it also explains why the KV cache is not multiplied the same way. common/arg.cpp sets params.n_parallel = -1 for the server example specifically; tools/server/server.cpp resolves that sentinel with params.n_parallel = 4; params.kv_unified = true;; common/common.cpp then assigns cparams.n_seq_max = params.n_parallel. src/llama-model.cpp sizes the recurrent memory with std::max((uint32_t) 1, cparams.n_seq_max) on the branch that names LLM_ARCH_QWEN35 explicitly, and src/llama-memory-recurrent.cpp allocates n_rows = mem_size * (1 + n_rs_seq), where n_rs_seq defaults to 0. So the recurrent state is allocated once per slot — four times. The same default line sets kv_unified = true, which gives all four slots one shared attention cache, so the 32 KiB/token figure above is not multiplied. One term scales with the slot count and the other does not; a budget that treats them alike is wrong in one direction or the other.

The whole trade, in one table. Accountable allocations (BF16 text weights + f16 projector + KV + recurrent state), against a 24 GiB card. Each cell is the sum of the exact byte-derived inputs, rounded once at the end:

Context	Weights	KV (f16)	Recurrent state	Sum	Left over
32,768	17.545	1.000	0.196	18.741	5.3
65,536	17.545	2.000	0.196	19.741	4.3
131,072	17.545	4.000	0.196	21.741	2.3
262,144 (native)	17.545	8.000	0.196	25.741	— does not fit

The native 262,144-token window is the first thing this card cannot have: 25.741 GiB is over 24 GiB outright, so unlike the 4B, the 9B genuinely trades context against nothing else at the top of the ladder. Everything at or below 131,072 fits.

The size question this card actually poses

On 12 GB the interesting question is which quant. Here it is which model, because on a 24 GB card the 9B's unquantised weights and the 27B's 4-bit weights land within a few hundred megabytes of each other — 16.690 GiB for Fara1.5-9B-bf16.gguf against 16.329 GiB (17,533,552,448 B) for Fara1.5-27B-Q4_K_M.gguf. Two very different models at nearly the same price.

The model card's own evaluation table is the only public evidence either way:

Model	WebVoyager	Online-Mind2Web	WebTailBench
Fara1.5-4B	80.8	57.3	27.4
Fara1.5-9B	86.6	63.4	32.3
Fara1.5-27B	89.3	72.3	40.2

Read what that table is and is not. Those scores are for the BF16 models — Microsoft evaluated the weights this recipe runs, and did not evaluate a 4-bit anything. So the choice on a 24 GB card is between a configuration whose numbers are published and a larger model whose published numbers do not describe the file you would actually load. Nobody has measured what Q4_K_M costs a model that emits pixel coordinates, and the gap it would have to survive on WebTailBench is 7.9 points. This recipe takes the 9B at the precision that was measured; the 27B at Q4_K_M is a defensible other answer, and it is a different page.

Why BF16 rather than a quant

Fara's output is pixel coordinates — the card describes its grounding data as "Curated datasets for predicting actions and pixel coordinates from screenshots, with images, text, and bounding boxes." Quantisation error lands on exactly that faculty, and no published evaluation measures how much grounding accuracy any tier costs. On 12 GB Q5_K_M is the honest ceiling; here the question disappears, and removing an unmeasured variable is worth more than a window nothing will fill.

BF16 is also a first-class type on this backend rather than a tolerated one, which is worth establishing before you download nineteen gigabytes of it:

• The hardware predicate. bf16_mma_hardware_available() in ggml/src/ggml-cuda/common.cuh returns true for GGML_CUDA_CC_IS_NVIDIA(cc) && cc >= GGML_CUDA_CC_AMPERE, and the same file defines GGML_CUDA_CC_AMPERE as 800. This card reports 860, so it clears the runtime's own BF16 tensor-core check — Ampere is the generation where that first became true on GeForce silicon.
• The decode kernel. ggml/src/ggml-cuda/mmvf.cu has explicit case GGML_TYPE_BF16 branches and reads weights through nv_bfloat162 pairs. Single-token generation from BF16 weights runs a purpose-built matrix-vector kernel; it is not converted to F16 first.
• The batched path. Prompt processing dispatches ggml_cuda_mul_mat_cublas_impl<GGML_TYPE_BF16> in ggml/src/ggml-cuda/ggml-cuda.cu, whose traits map BF16 to CUDA_R_16BF with F32 accumulation.

What BF16 costs, stated plainly. The BF16 file is 1.877x the bytes of the Q8_0 one (17,920,697,216 / 9,545,983,104), and single-token decode on a model this size is bound by how many weight bytes leave memory per token — so this is the slower configuration, and on a 9B the gap is larger in absolute terms than it was on the 4B. Nobody has measured Fara1.5 on any consumer GPU, so this recipe publishes a direction and not a ratio. If throughput matters more to you than the open question about quantised grounding, the same repo's ladder is a one-word substitution in Step 2 and Step 3 and changes nothing else on this page — accountable totals at the 65,536-token window below: Fara1.5-9B-Q8_0.gguf 8.890 GiB of weights, 11.942 total; Q6_K 7.171 / 10.223; Q5_K_M 6.382 / 9.434; Q4_K_M 5.505 / 8.556.

Why the f16 projector next to BF16 text weights. This one deserves an actual answer rather than a habit, because the argument that picks BF16 for the text weights — run the dtype the vendor evaluated — points straight at mmproj-bf16, and this recipe does not take it. Start from which build is which, because config.json will mislead you here: it declares "dtype": "bfloat16" under text_config and "dtype": "float32" under vision_config, but that field records an intent, not the contents of the repo. Read the safetensors headers instead and the vision tower ships exactly like the rest of the model — 333 vision tensors, every one BF16. So mmproj-Fara1.5-9B-bf16.gguf is the byte-faithful conversion and mmproj-Fara1.5-9B-f16.gguf is a re-encode, and taking the re-encode is a deliberate trade rather than a free one. Two things buy it. Precision at equal width: IEEE binary16 spends three more mantissa bits than bfloat16 over a narrower exponent range, and a vision tower's activations are range-bounded — the finer grid is worth more than the headroom on the stage that turns patch embeddings into the coordinates this model clicks on. And comparability: every accountable total on this page, and on every other Fara1.5 recipe in this catalogue, is computed with the f16 projector, so the budgets line up card to card. Neither of those is a measurement. If you would rather run the vendor's bytes, mmproj-Fara1.5-9B-bf16.gguf is a one-word substitution costing 3,538,944 B — 3.375 MiB, which moves the 65,536 total from 19.741 to 19.745. Nobody has measured which grounds better.

Why 65,536 tokens and not 131,072

131,072 fits — 21.741 GiB, with 2.3 GiB left — and the 24 GB recipes for the 4B in this catalogue do lead it, on an argument verified in the harness source: the per-round user text is bounded and tiny (the URL line is trimmed to MAX_URL_LENGTH = 100, the standing instruction is one fixed sentence, and the post-action observation is capped at max_observation_chars: int = 1000 and only produced for one action type — _TEXT_OBSERVATION_ACTIONS is the single-element frozenset {"read_page_answer_question"}), so the only genuinely unbounded term is the model's own reasoning plus tool call, once per round. The image half is capped outright at max_n_images: int = 3 in fara15_agent.py, which at the derived 1,260 tokens per screenshot pins images at 3,780 tokens for a whole run.

That argument about the harness is unchanged at 9B. What changed is the card. fara-cli defaults to --max_rounds 100 (run_fara.py), so:

 65,536 - 3,780 image tokens =  61,756 for text / 100 rounds ~   618 tokens per round
131,072 - 3,780             = 127,292 for text / 100 rounds ~ 1,270 tokens per round

On the 4B, buying the second row cost 2.000 GiB out of an 11.5 GiB surplus — under a fifth of the headroom, and the honest summary was that the card would not notice. Here it costs the same 2.000 GiB out of 4.3 GiB, i.e. roughly half of everything left, and this page does not derive what lives in that remainder: the CUDA context, the text graph's compute buffers, and the vision encoder's working set over a 5,040-patch image. This card would notice. So the default here is 65,536 — still more than four times the harness's documented 15,000-token floor, and enough for ~618 tokens of reasoning and tool call per round across a hundred rounds.

The honest counterweight, since Fara emits a chain-of-thought block before every action: 618 tokens per round is a budget, not a guarantee, and context shift is off by default (ctx_shift = false in common/common.h), so an overrun is a hard error rather than a silent truncation of the oldest turns. If you raise --max_rounds, or you watch a run and see long <think> traces, -c 131072 is a one-word change costing exactly 2.000 GiB and it still fits this card. Size the window for the run you intend. Going the other way, -c 32768 (18.741 GiB) is what the 12 GB tier leads and works fine here too.

Nothing in the table above is free space you can plan against to the last byte. Read the CUDA0 compute buffer size lines llama-server prints on startup for the real figures on your build, watch nvidia-smi with the agent actually running, and report what you see.

How the context is actually spent

A CUA sends a screenshot every step, so the per-image cost decides whether the budget above is realistic. It is not the number implied by the projector's position-embedding grid: llama.cpp treats this projector as dynamic-resolution, and --image-min-tokens / --image-max-tokens are documented as "only used by vision models with dynamic resolution".

The harness fixes the browser viewport at 1440x900 (viewport_width: int = 1440, viewport_height: int = 900 in fara15_agent.py), matching the card's note that "The screen resolution Fara is most commonly trained with is" 1440x900. It pre-resizes with Qwen's smart_resize at patch size 16 and merge size 2, so the factor is 32 and 900 becomes 896. llama.cpp then counts merged patches per axis in tools/mtmd/clip.cpp, where the PROJECTOR_TYPE_QWEN3VL branch returns (img->nx() / params.patch_size) / 2 across and (img->ny() / params.patch_size) / 2 down:

across : (1440 / 16) / 2 = 90 / 2 = 45
down   : ( 896 / 16) / 2 = 56 / 2 = 28
tokens : 45 x 28                 = 1,260 per screenshot
x max_n_images = 3               = 3,780 tokens resident

The source config.json's vision_config agrees on the geometry (patch_size: 16, spatial_merge_size: 2, depth: 27, out_hidden_size: 4096). The harness's own conservative default is image_token_estimate: int = 1500, a useful sanity check on the arithmetic, and the model card agrees on the retention: "We only keep the most recent 3 screenshots" in the chat history.

So images are capped and the text trajectory is what grows — which is the derivation the window decision above rests on.

Installation

1. Build llama.cpp with CUDA for sm_86

Ampere GA102 is compute capability 8.6. There is no special toolkit requirement here: 86-real sits in llama.cpp's default architecture list in ggml/src/ggml-cuda/CMakeLists.txt with no version gate around it, unlike the 89-real entry (guarded by CUDAToolkit_VERSION VERSION_GREATER_EQUAL "11.8") and the 120a-real entry (guarded by "12.8"). Naming the architecture explicitly still costs nothing and makes a mismatch a build-time failure rather than an idle GPU:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

Build from a release at or after b10087 — the release the GGUF used here was quantized on.

Both halves of the model are first-class in mainline — worth confirming before you download 19 GB:

grep -n 'LLM_ARCH_QWEN35,' src/llama-arch.cpp        # -> { LLM_ARCH_QWEN35, "qwen35" }
grep -n 'qwen3vl_merger' tools/mtmd/clip-impl.h      # -> PROJECTOR_TYPE_QWEN3VL

2. Download the weights and the projector

Both files are required. The projector is a separate artifact — downloading only the text GGUF gives you a model that cannot see.

pip install -U huggingface_hub

hf download bartowski/Fara1.5-9B-GGUF \
  Fara1.5-9B-bf16.gguf mmproj-Fara1.5-9B-f16.gguf \
  --local-dir ~/models/fara1.5-9b

That is 18.84 GB over the wire and it is the largest download in this model's catalogue apart from the 27B, so start it before you read the rest of the page.

Microsoft publishes no first-party GGUF for Fara1.5 — the microsoft org ships Fara1.5-4B, Fara1.5-9B and Fara1.5-27B as safetensors only. Four independent community quantisers have converted the 9B including its vision tower — bartowski, prithivMLmods, DevQuasar and Tdamre each publish a matching mmproj. Four independent conversions succeeding is the strongest available evidence that the vision path converts cleanly; it is still not a runtime test.

3. Serve it

./build/bin/llama-server \
  -m ~/models/fara1.5-9b/Fara1.5-9B-bf16.gguf \
  --mmproj ~/models/fara1.5-9b/mmproj-Fara1.5-9B-f16.gguf \
  -ngl 99 -c 65536 --host 127.0.0.1 --port 5000 \
  --image-min-tokens 1024 \
  --reasoning-format none \
  --temp 0

Three of those flags are load-bearing and none of them is a default:

• --reasoning-format none — mandatory. Without it the agent loop silently receives nothing; see Troubleshooting for the mechanism.
• --image-min-tokens 1024 — this projector ships no minimum-pixel key, so llama.cpp falls back to the floor set by hparams.set_limit_image_tokens(8, 4096) in the PROJECTOR_TYPE_QWEN3VL branch of tools/mtmd/clip.cpp and warns, a few lines later in the same branch: "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks". Fara is a grounding model, so that warning is aimed precisely at this case. At the harness's viewport the image is already 1,260 tokens and the floor never binds; it protects you if you shrink the viewport.
• -c 65536 — see the budget table and the window discussion above. This is a deliberate choice sized to --max_rounds 100, not the model's native maximum, which does not fit this card.

4. Prove the vision path works before going further

Check that the server advertises multimodal support, then send a real image. The server docs make the first part a client-side requirement rather than an optional nicety: "Clients should check /models or /v1/models for the multimodal capability before a multimodal request." Run GET /props alongside it — it reports which modalities loaded rather than merely that some did, the more useful signal when a projector silently failed to load.

curl -s http://127.0.0.1:5000/v1/models | grep -o multimodal

# supplementary — names the modality that actually loaded
curl -s http://127.0.0.1:5000/props | python3 -c "import sys,json; print(json.load(sys.stdin)['modalities'])"
# expect: {'vision': True}

curl -s http://127.0.0.1:5000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "fara",
  "temperature": 0,
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "Describe this image in one sentence."},
    {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}
  ]}]
}' | python3 -m json.tool

In the startup log, also find the clip_ctx: CLIP using <backend> backend line — clip.cpp emits it as "CLIP using %s backend" with the backend's own name, and CPU there means the projector is not on the GPU at all. If the reply describes the picture, the projector is wired up. If /v1/models never prints multimodal, if /props reports vision: False, or if the model answers as though no image were present, stop here — the agent loop below will produce confidently wrong actions rather than fail loudly.

5. Install the harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Skip the README's [vllm] extra — that path serves BF16 safetensors on datacenter cards, which is not how this recipe gets to BF16.

Running

Two clients share the same llama-server endpoint. Path A (fara-cli) is the smoke test, with no isolation. Path B (Magentic-UI) is what the vendor recommends for anything real. They are not equivalent in risk; pick deliberately.

Path A — fara-cli, for a first smoke test

fara-cli speaks to any OpenAI-compatible endpoint, and this is the one size where its built-in defaults already match: DEFAULT_ENDPOINT_CONFIG in run_fara.py is {"model": "Fara1.5-9B", "base_url": "http://localhost:5000/v1", ...}. Name them explicitly anyway, so the command does not depend on a default that could move:

source .venv/bin/activate
fara-cli \
  --base_url http://127.0.0.1:5000/v1 \
  --api_key sk-no-key-required \
  --model fara \
  --task "find the current UTC time on time.is and tell me" \
  --output_folder ./runs/first \
  --save_screenshots \
  --headful \
  --max_rounds 15

--headful is in that command on purpose. The harness runs the browser headless by default — run_fara.py declares the flag with the help text "Run the browser in headful mode (show GUI, default is headless)" and both launch sites read headless=not args.headful. Either way this drives a real, unisolated Chromium on your machine; the danger is identical with or without the flag. What the flag changes is whether you can see it, and the supervision advice above assumes you can — so it is what makes that advice true rather than aspirational. Drop it once you trust the loop and are reading the trajectory files instead.

Each step writes a screenshot and a data_point.json into --output_folder, which is how you tell a grounding failure (clicks landing in the wrong place) from a reasoning failure. Note the low --max_rounds 15 above: the budget on this page is sized for the default 100, but a first run should be short.

Keep the task on a low-stakes public site and stay signed out. Fara is trained to stop and ask before entering personal information, making payments, submitting forms, signing in or sending messages; if you see it sail through one of those, stop the run — that is a wiring problem in your prompt path, not a quirk to tolerate.

Path B — Magentic-UI, for the container

Everything the model card assumes — the containerised browser, domain allow-lists, watch-mode and a pause control — lives in Magentic-UI, not in fara-cli. That is also where a Docker requirement would come from; there is no Docker step in Path A because Path A genuinely does not use one.

Magentic-UI takes the same OpenAI-compatible endpoint you started above, so nothing on this page changes except the client. Its setup is documented in its own repository rather than reproduced here, because this recipe has not verified those steps end to end on this GPU — give it http://127.0.0.1:5000/v1 as the model endpoint. Whichever client you use, match the trained resolution: the card is explicit — "Match this in your sandbox for the most reliable grounding."

Results

• Speed: omitted — no published measurement of Fara1.5-9B exists for this card or for any consumer GPU, and the only hardware Microsoft names is datacenter class. Borrowing a figure from an A100 for a 24 GB Ampere card would be misleading, and the vendor publishes no throughput numbers to borrow in the first place. Any 24 GB sm_86 card lands on the same memory budget as this one, because the derivation above depends only on capacity and compute capability — but memory bandwidth is exactly what those cards do not share, and with no measurement on any of them this recipe does not rank them. If you run this, please contribute your numbers so this line can be replaced with a measurement — and if you run both weight tiers, the BF16-versus-Q8_0 step latency is the most useful pair of numbers anyone could add to this page.
• VRAM usage: 17.545 GiB of weights and projector; 19.741 GiB accountable including the f16 KV cache and the four-slot recurrent state at a 65,536-token context, derived above from byte counts and llama.cpp's memory formulas. This is a derivation, not an observed peak — compute buffers, the vision graph and the browser's own GPU memory sit on top. Track the empirical picture at /check/fara1-5-9b/rtx-3090.
• Quality notes: the vendor's published scores for this size — 86.6 WebVoyager, 63.4 Online-Mind2Web, 32.3 WebTailBench, per the model card's evaluation table — are for the BF16 model on datacenter hardware. The hardware does not transfer; the precision does, which is the point of leading BF16 here. No public evaluation measures what quantisation costs a model whose job is predicting exact pixel coordinates, and on this card you do not have to find out.
• Language: English only — the card puts "Languages other than English (training data is English-only)" out of scope.

For the full benchmark data, see /check/fara1-5-9b/rtx-3090.

Troubleshooting

The agent does nothing — every step logs an empty response or list index out of range

This is the failure everyone hits, and it is not a bug in your setup. Fara opens a <think> block and never closes it — you can read that straight out of the chat template embedded in the GGUF you downloaded, whose generation prompt emits '<think>\n' with no closing tag unless enable_thinking is explicitly false. Under llama-server's default reasoning extraction the parser therefore treats the whole output — reasoning and the <tool_call> block together — as reasoning, returns it in reasoning_content, and leaves message.content empty. The harness parses message.content for <tool_call>, finds nothing, and the loop dies on round one.

Launch with --reasoning-format none, documented as "none: leaves thoughts unparsed in message.content". The raw output then begins with a stray unclosed <think> tag, which is cosmetic.

Reported for the 27B sibling by a community user in the model family's HF discussions and acknowledged there by Microsoft — swhitehead, whose HF account carries the org-member flag on that repo, replied "Would you mind posting this as an issue on the Fara1.5 github repo so we can mention this in the hosting instructions?", which became microsoft/fara#82 — still open at the time of writing, titled "Hosting: llama.cpp needs --reasoning-format none". It applies to the 9B for a checkable reason rather than by assumption: the 9B's own GGUF carries the same chat template, whose generation prompt emits the unclosed <think>. The model card documents the output shape it comes from — "Chain-of-thought block followed by a tool-call block (XML-tagged)".

The model answers but ignores the screenshot

Almost always the projector. Confirm --mmproj is on the command line, that the server logged an mmproj load stage and a CLIP using … line, that /v1/models advertises the multimodal capability, and that Step 4's /props call reports vision: True. A text-only GGUF with no projector loads and runs perfectly happily — it simply cannot see, which in a CUA presents as confidently wrong actions rather than an error.

Clicks land in the wrong place

Check the viewport first: the harness defaults to 1440x900 because that is what the model was mostly trained at, and a different window size changes both the grounding distribution and the token cost derived above. Then confirm --image-min-tokens 1024 is set, since llama.cpp's own warning ties sub-1024-token images to degraded grounding. Only after both should you suspect the weights — and on the lead configuration here you are running the unquantised BF16 conversion of the weights the vendor evaluated, so the quant ladder is not available as an explanation at all. That is a diagnostic advantage of this card worth keeping: it removes one variable from every grounding bug you file.

Out of memory at load

19.741 GiB accountable against 24 GiB is a real but not generous margin on a card that also drives your display, and this is the tier where that starts to matter — the Chromium this agent opens competes for the same 24 GB. In order of what to try:

1. Drop -c from 65536 to 32768, which frees exactly 1.000 GiB and is still more than twice the harness's 15,000-token floor.
2. Quantise the K cache only: --cache-type-k q8_0. block_q8_0 is one ggml_half scale plus 32 int8_t quants — 34 bytes per 32 values against f16's 64 — so the K half drops from 16 to 8.5 KiB/token and the total from 32 to 24.5 KiB/token, taking the 65,536-token cache from 2.000 GiB to 1.531 GiB.
3. Step down one weight tier. From BF16 the first rung is enormous: Fara1.5-9B-Q8_0.gguf frees 7.800 GiB, which takes the whole budget to 11.942 GiB and ends the conversation. If you get here, you have also re-opened the quantised-grounding question this page exists to close, so treat it as a fallback rather than a tune.
4. --no-mmproj-offload keeps the projector on the CPU, freeing 0.855 GiB at the cost of slower image encoding — once per agent turn.

Don't reach for speculative decoding to fill the headroom

The model family carries a multi-token-prediction layer — the source config.json declares mtp_num_hidden_layers: 1 inside text_config. That layer is not part of the GGUF conversion path used here: llama.cpp names the corresponding header key nextn_predict_layers (LLM_KV_NEXTN_PREDICT_LAYERS in src/llama-arch.cpp), and the file this recipe downloads carries no blk.*.nextn.* tensors for --spec-type to attach to. Nothing on this page enables it; if you go looking, that is why it does nothing.

rope.dimension_sections has wrong array length; expected 4, got 3

This is a real open llama.cpp bug (PR #25334, titled "fix: load qwen35 Ollama GGUF exports"), but it belongs to Ollama-exported qwen35 blobs, not to the GGUF this recipe uses. The source config's rope_parameters.mrope_section is [11, 11, 10], which the bartowski conversion writes with the trailing zero llama.cpp expects. If you see this error, you are loading a different file than the one downloaded in Step 2.

ollama run fara does nothing — there is no Ollama entry

There is no Ollama library entry for Fara1.5 in any form. The GGUF / LM Studio / Ollama line in the harness README refers to Fara-7B, the previous generation, reached with a --fara-7b flag — it is not about Fara1.5. Likewise, the "run them in your choice of tools" list on the quantiser's card is that quantiser's standard boilerplate. llama.cpp built from source is the path this recipe documents.

Generation is slower than you expected

Two things to separate before you go bug-hunting. The first is expected: BF16 moves 1.877x the bytes per token that Q8_0 does, and on a 9B that is nearly eight gigabytes of extra traffic per token-generation pass — a cost you chose in exchange for skipping the quantisation question. A/B against Fara1.5-9B-Q8_0.gguf with llama-bench before treating it as a defect. The second is a lead rather than a known property: llama.cpp issue #25162 is an open, community-reported regression against exactly the SSM/linear-attention kernels this architecture leans on, titled "Performance regression on Turing GPUs after 9e58d4d69 — Qwen35 SSM kernels affected". Read its scope carefully: the title says Turing, the reporter measured on an RTX 2080 Ti, the thread's two comments are both from community accounts with no maintainer response, and nobody has posted numbers for an Ampere card. It is a thing to check with llama-bench across a couple of builds, not a property of this one.