Fara1.5-4B on RTX 3090 Ti: BF16 Browser Computer-Use Agent on Ampere sm_86

What You'll Build

A local browser computer-use agent (CUA): Microsoft's Fara1.5-4B served by llama-server on an RTX 3090 Ti, with the vision projector loaded so the model actually sees browser screenshots, driven by Microsoft's own fara-cli harness against a plain local Playwright Chromium.

Every smaller card in this catalogue has to answer a question nobody has measured: how much grounding accuracy does quantisation cost a model whose output is pixel coordinates? On 24 GB you decline the question — and for the first time you also stop having to trade it against context length. The lead configuration below is the unquantised BF16 conversion of the weights Microsoft actually evaluated, at a 131,072-token window, and it still leaves more than eleven gigabytes of the card unspoken for.

Hardware data: RTX 3090 Ti (24GB VRAM) · BF16 weights + f16 mmproj = 8.472 GiB, 12.668 GiB accountable at 131,072 context (derived below) · See benchmark data

⚠️ This model clicks, types and submits in a real browser, and the path documented here gives it no isolation of any kind. That is the operating requirement, not boilerplate, and it is the thing to settle before anything else on this page.

The model card is blunt about the consequence: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." That sentence closes the card's Recommended Deployment: MagenticLite section, which lists four controls as MagenticLite features — the first being "the browser runs in a Docker container with no access to host files or environment variables" — and then says: "If you integrate Fara1.5-4B directly, you're responsible for these controls." Fara also reads whatever is on the page, so a hostile page is a prompt-injection surface.

fara-cli is a direct integration. Its install step is playwright install, and it drives a plain local Playwright Chromium — no container, no allow-list, no pause control. The harness README offers the containerised option as an alternative to it rather than as part of it: "instead of fara-cli you can use Magentic-UI", described in the same sentence as "a sandboxed browser environment with auditable action logging and user prompts at critical points". For the CLI path the vendor's position is a recommendation, not something shipped: "We recommend running it in a sandboxed environment, monitoring its execution, and avoiding sensitive data or high-risk domains."

So on the fara-cli path you are the boundary. Practically: a throwaway OS user or a VM, Playwright's own fresh browser profile (never your daily one), signed out of everything, low-stakes public sites, and eyes on the run rather than leaving it unattended. The command in Running passes --headful for exactly that last reason. If you want the container, the allow-lists, watch-mode and the pause control, use Magentic-UI as the client instead.

ℹ️ Vision is the product, not a bonus. Per the model card, "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree." A Fara install that only serves text is a broken install — the --mmproj step below is mandatory, and Step 4 exists to prove that an image really reached the model before you let it drive anything.

⚠️ Runtime status — read before you commit an evening to this. llama.cpp support for this architecture is verifiable statically, and this recipe shows that evidence rather than asserting it. What does not exist anywhere public is a report of Fara1.5 decoding an image under llama-server or llama-mtmd-cli, on this card or on any other. Nor is there a throughput measurement for this model on any consumer GPU — see Results for why there is no speed figure here. Treat your first run as the test, and use Step 4, which reads llama.cpp's own load-time self-report rather than asking you to trust this page.

Requirements

Component	Minimum	This recipe
GPU	16GB VRAM, CUDA compute 8.0+ (Ampere or newer, for BF16 tensor cores)	RTX 3090 Ti (24GB) — not measured; the budget below is derived (contribute a run)
RAM	16GB (weights are mmap'd on their way to VRAM; Playwright's Chromium sits on top)	—
Storage	9.10 GB (BF16 + f16 mmproj)	9,096,817,152 B = 8.472 GiB, plus the harness and Chromium
Software	llama.cpp b10087+ (CUDA), Python 3.10+, Playwright	—

Two notes on that table. min_vram_gb is 16, not 24 — it is the filter floor for the install documented here, not this card's capacity. The BF16 configuration below accounts for 12.668 GiB, which clears a 16 GB card; what a 16 GB card does not get is the eleven-plus gigabytes of undivided headroom this one has, which is why the 16 GB recipes for this model lead a shorter window. And the GPU cell is not a measurement — /check reports unknown with zero benchmark rows, and nothing on this page was run on hardware by us.

Microsoft ships safetensors only, and all of it in BF16: the two shards total 8.455 GiB per the HF tree API (5,960,243,904 + 3,118,376,552 B), and the repo's own config.json declares "dtype": "bfloat16". Its Quickstart serves those with vLLM (--dtype bfloat16) and names datacenter hardware for the job: "A GPU with enough memory for a 4B model in bf16 (A6000, A100, H100, and B200 have been tested)", and the card's highlights claim only that the model "runs on a single A100/H100/B200 with room for the screenshot history". What this recipe shows is that the precision in that sentence is available on a 24 GB consumer card, with the window to go with it.

What 24 GB actually buys

There is no benchmark for this pair, so every number below is arithmetic from the artifacts' own metadata and llama.cpp's source, not a measurement. Reproduce it rather than trusting it. All totals are GiB (2^30 bytes), the unit a "24GB" NVIDIA card is really in, so they compare directly against 24.

Weights. Byte counts from the bartowski GGUF tree:

Component	Bytes	GiB
Fara1.5-4B-bf16.gguf	8,424,393,568	7.846
mmproj-Fara1.5-4B-f16.gguf	672,423,584	0.626
Weights total	9,096,817,152	8.472

KV cache — and this is where Fara1.5 is unusual. Its qwen35 architecture is a hybrid: llm_arch_is_hybrid() in src/llama-arch.cpp returns true for LLM_ARCH_QWEN35, and src/models/qwen35.cpp marks layers recurrent off the full_attention_interval. The BF16 file's own header carries qwen35.block_count = 32 and qwen35.full_attention_interval = 4, and the source config.json spells the split out layer by layer: its layer_types array holds 24 linear_attention entries and only 8 full_attention. Only those 8 carry a cache that grows with context.

With num_key_value_heads = 4 and head_dim = 256 (qwen35.attention.head_count_kv and attention.key_length / value_length in the GGUF):

K+V per token per attention layer = 2 x 4 x 256 x 2 B (f16) = 4,096 B
x 8 full-attention layers                                   = 32 KiB / token

A non-hybrid 32-layer model with the same head geometry would cost four times that. Context on this model is cheap — doubling -c costs exactly 1.000 GiB per doubling from 32,768 — and on a 24 GB card that is what changes the shape of the decision.

Recurrent state — fixed, and tiny. The other 24 layers hold a context-independent state, sized by n_embd_r() / n_embd_s() in src/llama-hparams.cpp from the artifact's own GGUF keys (ssm.conv_kernel 4, ssm.inner_size 4096, ssm.group_count 16, ssm.state_size 128 — the config's linear_conv_kernel_dim, linear_num_value_heads x linear_value_head_dim, linear_num_key_heads and linear_key_head_dim respectively). qwen35 sets no n_embd_head_kda, so both fall through to the Mamba forms:

conv state : (4-1) x (4096 + 2 x 16 x 128) =  24,576 elems
ssm  state :         128 x 4096            = 524,288 elems
per layer  : 548,864 x 4 B (F32)           = 2.094 MiB
x 24 layers                                = 50.25 MiB   <- per sequence, does not grow with context
x 4 sequences (llama-server's default)     = 201 MiB = 0.196 GiB

That last multiplication is easy to miss and it is not optional. src/llama-model.cpp sizes the recurrent state with /* recurrent_rs_size */ std::max((uint32_t) 1, cparams.n_seq_max) on the branch that names LLM_ARCH_QWEN35, cparams.n_seq_max comes from params.n_parallel, and tools/server/server.cpp resolves the server's auto default with params.n_parallel = 4 — so the allocation is four sequences' worth whether or not you use them. The KV cache is not multiplied: the same default also sets kv_unified = true, so the four slots share one context window rather than each getting their own.

The whole trade, in one table. Accountable allocations (BF16 text weights + f16 projector + KV + recurrent state), against a 24 GiB card:

Context	Weights	KV (f16)	Recurrent state	Sum	Left over
32,768	8.472	1.000	0.196	9.668	14.3
65,536	8.472	2.000	0.196	10.668	13.3
131,072	8.472	4.000	0.196	12.668	11.3
262,144 (native)	8.472	8.000	0.196	16.668	7.3

Every row fits. That is the whole difference from the smaller cards in this catalogue, and it means the interesting question on this page is not which quant — it is how much window is worth buying, which is a different question and has a less flattering answer than the card's capacity suggests.

Why not the full 262,144-token window

It is affordable here: 16.668 GiB, with 7.3 GiB spare. It is still not what this recipe leads with, and the reason is the harness rather than the card.

The image half of the prompt is capped, not growing: max_n_images: int = 3 in fara15_agent.py, which at the derived 1,260 tokens per screenshot (below) pins images at 3,780 tokens for the whole run. What grows is the text trajectory — and its per-round contribution is bounded on the user side too: the observation appended after an action is capped at max_observation_chars: int = 1000 and only produced for one action type (_TEXT_OBSERVATION_ACTIONS is the single-element frozenset {"read_page_answer_question"}), the URL line is trimmed to MAX_URL_LENGTH = 100, and the standing instruction is one fixed sentence. The genuinely unbounded term is the model's own reasoning plus tool call, once per round.

fara-cli defaults to --max_rounds 100 (run_fara.py). So divide:

131,072 - 3,780 image tokens = 127,292 for text / 100 rounds ~ 1,270 tokens per round
262,144 - 3,780              = 258,364 for text / 100 rounds ~ 2,580 tokens per round

— before subtracting the system prompt, which this recipe does not measure. Nobody has published the distribution of Fara's per-step reasoning length, so the honest statement is that ~1,270 tokens of budget per round is generous for a reasoning block plus a <tool_call> JSON object, and ~2,580 is generous twice over. Buying the second one costs 4 GiB of KV that the default run has no way to fill. The harness README asks for a floor, not a ceiling — "Please ensure that context length is set to at least 15000 tokens and temperature to 0 for best results." — and Microsoft's own framing of the 262K window is "Long enough for multi-screenshot trajectories with full action history.", which 131,072 already satisfies for a hundred rounds.

Two caveats that keep this honest. First, this is the argument the 16 GB recipes for this model make as well, and it does not get weaker because a bigger card can pay: an empty window is the same empty window. Second, if you genuinely run past a hundred rounds, or raise --max_rounds yourself, then -c 262144 is a one-word change and it fits — that is a real thing 24 GB buys, and the reason to state it rather than hide it. Note that context shift is off by default (ctx_shift = false in common/common.h), so the server does not quietly discard the oldest tokens of a trajectory to make room; size the window for the run you intend.

Going the other way, -c 65536 (10.668 GiB) is what the 16 GB tier leads and is also fine here; it simply leaves 13.3 GiB of a 24 GB card idle instead of 11.3.

Why BF16 rather than a quant

Fara's output is pixel coordinates — the card describes the training data as "Curated datasets for predicting actions and pixel coordinates from screenshots, with images, text, and bounding boxes." Quantisation error lands on exactly that faculty, and no published evaluation measures how much grounding accuracy any tier costs. The vendor's published scores are for BF16. On 8 GB there is no choice; on 12 GB Q8_0 is the honest ceiling; here the question disappears, and removing an unmeasured variable is worth more than a window nothing will fill.

BF16 is also a first-class type on this backend rather than a tolerated one, which is worth establishing before you download nine gigabytes of it:

• The hardware predicate. bf16_mma_hardware_available() in ggml/src/ggml-cuda/common.cuh returns true for GGML_CUDA_CC_IS_NVIDIA(cc) && cc >= GGML_CUDA_CC_AMPERE, and the same file defines GGML_CUDA_CC_AMPERE as 800. This card reports 860, so it clears the runtime's own BF16 tensor-core check — Ampere is the generation where that first became true on GeForce silicon.
• The decode kernel. ggml/src/ggml-cuda/mmvf.cu has explicit case GGML_TYPE_BF16 branches and reads weights through nv_bfloat162 pairs. Single-token generation from BF16 weights runs a purpose-built matrix-vector kernel; it is not converted to F16 first.
• The batched path. Prompt processing dispatches ggml_cuda_mul_mat_cublas_impl<GGML_TYPE_BF16> in ggml/src/ggml-cuda/ggml-cuda.cu, whose traits map BF16 to CUDA_R_16BF with F32 accumulation.

The artifact matches: reading the header of the exact file downloaded in Step 2 gives general.architecture = qwen35 and general.file_type = 32 (LLAMA_FTYPE_MOSTLY_BF16), with a census of 426 tensors spanning blk.0 through blk.31 and no gaps.

What BF16 costs, stated plainly. The BF16 file is 1.875x the bytes of the Q8_0 one (8,424,393,568 / 4,493,954,144), and single-token decode on a model this small is bound by how many weight bytes leave memory per token — so this is the slower configuration. Nobody has measured Fara1.5 on any consumer GPU, so this recipe publishes a direction and not a ratio. It is also 9.10 GB to download instead of 5.17 GB (the projector is the same file either way). If either matters more to you than the open question about quantised grounding, the same repo's ladder is a one-word substitution in Step 2 and Step 3 and changes nothing else on this page: Fara1.5-4B-Q8_0.gguf 4.185 GiB, Fara1.5-4B-Q6_K.gguf 3.425 GiB, Fara1.5-4B-Q5_K_M.gguf 3.088 GiB, Fara1.5-4B-Q4_K_M.gguf 2.687 GiB.

Why the f16 projector next to BF16 text weights. bartowski publishes both an f16 and a BF16 build of the vision tower and they differ by exactly 3 MiB (672,423,584 vs 675,569,312 B), so this is not a budget decision. At the same two bytes per value, IEEE binary16 spends three more bits on the mantissa than bfloat16 does and three fewer on the exponent — a finer grid over a narrower range — and the projector is the stage that turns patch embeddings into the coordinates this model clicks on. This recipe takes the f16 build, which is what every other recipe for this model uses. If you would rather keep the source dtype byte-for-byte, substitute mmproj-Fara1.5-4B-bf16.gguf and add 3 MiB to every figure above.

Nothing in the table above is free space you can plan against to the last byte: the CUDA context, compute buffers and the vision encoder's working set are allocated at load time and this recipe does not derive them. Read the CUDA0 compute buffer size lines llama-server prints on startup for the real figures on your build, watch nvidia-smi with the agent actually running, and report what you see.

How the context is actually spent

A CUA sends a screenshot every step, so the per-image cost decides whether the budget above is realistic. It is not the 768 implied by the projector's position-embedding grid: llama.cpp treats this projector as dynamic-resolution, and --image-min-tokens / --image-max-tokens are documented as "only used by vision models with dynamic resolution".

The harness fixes the browser viewport at 1440x900 (viewport_width: int = 1440, viewport_height: int = 900 in fara15_agent.py), matching the card's note that "The screen resolution Fara is most commonly trained with is" 1440x900. It pre-resizes with Qwen's smart_resize at patch size 16 and merge size 2, so the factor is 32 and 900 becomes 896. llama.cpp then counts merged patches per axis in tools/mtmd/clip.cpp, where the PROJECTOR_TYPE_QWEN3VL branch returns (img->nx() / params.patch_size) / 2 across and (img->ny() / params.patch_size) / 2 down:

across : (1440 / 16) / 2 = 90 / 2 = 45
down   : ( 896 / 16) / 2 = 56 / 2 = 28
tokens : 45 x 28                 = 1,260 per screenshot
x max_n_images = 3               = 3,780 tokens resident

The projector's own GGUF header agrees with the geometry (clip.vision.patch_size = 16, clip.vision.spatial_merge_size = 2), as does the source config.json's vision_config. The harness's own conservative default is image_token_estimate: int = 1500, a useful sanity check on the arithmetic, and the model card agrees on the retention: "We only keep the most recent 3 screenshots" in the chat history.

So images are capped and the text trajectory is what grows — which is the derivation the window decision above rests on.

Installation

1. Build llama.cpp with CUDA for sm_86

Ampere GA102 is compute capability 8.6. There is no special toolkit requirement here: 86-real sits in llama.cpp's default architecture list in ggml/src/ggml-cuda/CMakeLists.txt with no version gate around it, unlike the 89-real entry (guarded by CUDAToolkit_VERSION VERSION_GREATER_EQUAL "11.8") and the 120a-real entry (guarded by "12.8"). Naming the architecture explicitly still costs nothing and makes a mismatch a build-time failure rather than an idle GPU:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

Build from a release at or after b10087 — the release the GGUF used here was quantized on, published 2026-07-22.

Both halves of the model are first-class in mainline — worth confirming before you download 9 GB:

grep -n 'LLM_ARCH_QWEN35,' src/llama-arch.cpp        # -> { LLM_ARCH_QWEN35, "qwen35" }
grep -n 'qwen3vl_merger' tools/mtmd/clip-impl.h      # -> PROJECTOR_TYPE_QWEN3VL

2. Download the weights and the projector

Both files are required. The projector is a separate artifact — downloading only the text GGUF gives you a model that cannot see.

pip install -U huggingface_hub

hf download bartowski/Fara1.5-4B-GGUF \
  Fara1.5-4B-bf16.gguf mmproj-Fara1.5-4B-f16.gguf \
  --local-dir ~/models/fara1.5-4b

Microsoft publishes no first-party GGUF for Fara1.5 — the microsoft org ships Fara1.5-4B, Fara1.5-9B and Fara1.5-27B as safetensors, plus an ONNX repo for the previous-generation Fara-7B, which is a different model and should never be mixed in. Four independent community quantisers have converted the 4B including its vision tower — bartowski, prithivMLmods, DevQuasar and runanywhere each publish a matching mmproj. Four independent conversions succeeding is the strongest available evidence that the vision path converts cleanly; it is still not a runtime test. bartowski's card labels the file this recipe uses "Full BF16 weights."

3. Serve it

./build/bin/llama-server \
  -m ~/models/fara1.5-4b/Fara1.5-4B-bf16.gguf \
  --mmproj ~/models/fara1.5-4b/mmproj-Fara1.5-4B-f16.gguf \
  -ngl 99 -c 131072 --host 127.0.0.1 --port 5000 \
  --image-min-tokens 1024 \
  --reasoning-format none \
  --temp 0

Three of those flags are load-bearing and none of them is a default:

• --reasoning-format none — mandatory. Without it the agent loop silently receives nothing; see Troubleshooting for the mechanism.
• --image-min-tokens 1024 — this projector ships no minimum-pixel key, so llama.cpp falls back to the floor set by hparams.set_limit_image_tokens(8, 4096) in the PROJECTOR_TYPE_QWEN3VL branch of tools/mtmd/clip.cpp and warns, a few lines later in the same branch: "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks". Fara is a grounding model, so that warning is aimed precisely at this case. At the harness's viewport the image is already 1,260 tokens and the floor never binds; it protects you if you shrink the viewport.
• -c 131072 — see the budget table and the window discussion above. This is a deliberate choice sized to --max_rounds 100, not the model's native maximum.

4. Prove the vision path works before going further

Check that the server advertises multimodal support, then send a real image. The server docs make the first part a client-side requirement rather than an optional nicety: "Clients should check /models or /v1/models for the multimodal capability before a multimodal request." Run GET /props alongside it — it reports which modalities loaded rather than merely that some did, the more useful signal when a projector silently failed to load. For the image itself, image_url.url "can be a remote URL, base64 (raw or URI-encoded via data:image/...;base64) or path to local file".

curl -s http://127.0.0.1:5000/v1/models | grep -o multimodal

# supplementary — names the modality that actually loaded
curl -s http://127.0.0.1:5000/props | python3 -c "import sys,json; print(json.load(sys.stdin)['modalities'])"
# expect: {'vision': True}

curl -s http://127.0.0.1:5000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "fara",
  "temperature": 0,
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "Describe this image in one sentence."},
    {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}
  ]}]
}' | python3 -m json.tool

In the startup log, also find the clip_ctx: CLIP using <backend> backend line — clip.cpp emits it as "CLIP using %s backend" with the backend's own name, and CPU there means the projector is not on the GPU at all. If the reply describes the picture, the projector is wired up. If /v1/models never prints multimodal, if /props reports vision: False, or if the model answers as though no image were present, stop here — the agent loop below will produce confidently wrong actions rather than fail loudly.

5. Install the harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Skip the README's [vllm] extra — that path serves BF16 safetensors on datacenter cards, which is not how this recipe gets to BF16.

Running

Two clients share the same llama-server endpoint. Path A (fara-cli) is the smoke test, with no isolation. Path B (Magentic-UI) is what the vendor recommends for anything real. They are not equivalent in risk; pick deliberately.

Path A — fara-cli, for a first smoke test

fara-cli speaks to any OpenAI-compatible endpoint. Its built-in default already points at http://localhost:5000/v1, but name the model explicitly, since DEFAULT_ENDPOINT_CONFIG in run_fara.py names a different size ("model": "Fara1.5-9B"):

source .venv/bin/activate
fara-cli \
  --base_url http://127.0.0.1:5000/v1 \
  --api_key sk-no-key-required \
  --model fara \
  --task "find the current UTC time on time.is and tell me" \
  --output_folder ./runs/first \
  --save_screenshots \
  --headful \
  --max_rounds 15

--headful is in that command on purpose. The harness runs the browser headless by default — run_fara.py declares the flag with the help text "Run the browser in headful mode (show GUI, default is headless)" and both launch sites read headless=not args.headful. Either way this drives a real, unisolated Chromium on your machine; the danger is identical with or without the flag. What the flag changes is whether you can see it, and the supervision advice above assumes you can — so it is what makes that advice true rather than aspirational. Drop it once you trust the loop and are reading the trajectory files instead.

Each step writes a screenshot and a data_point.json into --output_folder, which is how you tell a grounding failure (clicks landing in the wrong place) from a reasoning failure. Note the low --max_rounds 15 above: the budget on this page is sized for the default 100, but a first run should be short.

Keep the task on a low-stakes public site and stay signed out. Fara is trained to stop and ask before entering personal information, making payments, submitting forms, signing in or sending messages; if you see it sail through one of those, stop the run — that is a wiring problem in your prompt path, not a quirk to tolerate.

Path B — Magentic-UI, for the container

Everything the model card assumes — the containerised browser, domain allow-lists, watch-mode and a pause control — lives in Magentic-UI, not in fara-cli. That is also where a Docker requirement would come from; there is no Docker step in Path A because Path A genuinely does not use one.

Magentic-UI takes the same OpenAI-compatible endpoint you started above, so nothing on this page changes except the client. Its setup is documented in its own repository rather than reproduced here, because this recipe has not verified those steps end to end on this GPU — give it http://127.0.0.1:5000/v1 as the model endpoint. Whichever client you use, match the trained resolution: the card is explicit — "Match this in your sandbox for the most reliable grounding."

Results

• Speed: omitted — no published measurement of Fara1.5-4B exists for this card or for any consumer GPU, and the only hardware Microsoft names is datacenter class. Borrowing a figure from an A100 for a 24 GB Ampere card would be misleading, and the vendor publishes no throughput numbers to borrow in the first place. Any 24 GB sm_86 card lands on the same memory budget as this one, because the derivation above depends only on capacity and compute capability — but memory bandwidth is exactly what those cards do not share, and with no measurement on any of them this recipe does not rank them. If you run this, please contribute your numbers so this line can be replaced with a measurement — and if you run both weight tiers, the BF16-versus-Q8_0 step latency is the most useful pair of numbers anyone could add to this page.
• VRAM usage: 8.472 GiB of weights and projector; 12.668 GiB accountable including KV and the four-sequence recurrent state at 131,072 context, derived above from the GGUF's own metadata and llama.cpp's memory formulas. This is a derivation, not an observed peak — compute buffers, the vision graph and the browser's own GPU memory sit on top. Track the empirical picture at /check/fara1-5-4b/rtx-3090-ti.
• Quality notes: the vendor's published scores for this size — 80.8 WebVoyager, 57.3 Online-Mind2Web, 27.4 WebTailBench, per the model card's evaluation table — are for the BF16 model on datacenter hardware. The hardware does not transfer; the precision does, which is the point of leading BF16 here. No public evaluation measures what quantisation costs a model whose job is predicting exact pixel coordinates, and on this card you do not have to find out.
• Language: English only — the card puts "Languages other than English (training data is English-only)" out of scope.

For the full benchmark data, see /check/fara1-5-4b/rtx-3090-ti.

Troubleshooting

The agent does nothing — every step logs an empty response or list index out of range

This is the failure everyone hits, and it is not a bug in your setup. Fara opens a <think> block and never closes it — you can read that straight out of the chat template embedded in the GGUF you downloaded, whose generation prompt emits '<think>\n' with no closing tag unless enable_thinking is explicitly false. Under llama-server's default reasoning extraction the parser therefore treats the whole output — reasoning and the <tool_call> block together — as reasoning, returns it in reasoning_content, and leaves message.content empty. The harness parses message.content for <tool_call>, finds nothing, and the loop dies on round one.

Launch with --reasoning-format none, documented as "none: leaves thoughts unparsed in message.content". The raw output then begins with a stray unclosed <think> tag, which is cosmetic.

Reported for the 27B sibling by a community user in the model family's HF discussions and acknowledged there by Microsoft — swhitehead, whose HF account carries the org-member flag on that repo, replied "Would you mind posting this as an issue on the Fara1.5 github repo so we can mention this in the hosting instructions?", which became microsoft/fara#82 — still open, titled "Hosting: llama.cpp needs --reasoning-format none". It applies to the 4B for a checkable reason rather than by assumption: the 4B's own GGUF carries the same chat template, whose generation prompt emits the unclosed <think>.

The model answers but ignores the screenshot

Almost always the projector. Confirm --mmproj is on the command line, that the server logged an mmproj load stage and a CLIP using … line, that /v1/models advertises the multimodal capability, and that Step 4's /props call reports vision: True. A text-only GGUF with no projector loads and runs perfectly happily — it simply cannot see, which in a CUA presents as confidently wrong actions rather than an error.

Clicks land in the wrong place

Check the viewport first: the harness defaults to 1440x900 because that is what the model was mostly trained at, and a different window size changes both the grounding distribution and the token cost derived above. Then confirm --image-min-tokens 1024 is set, since llama.cpp's own warning ties sub-1024-token images to degraded grounding. Only after both should you suspect the weights — and on the lead configuration here you are running the unquantised BF16 conversion of the weights the vendor evaluated, so the quant ladder is not available as an explanation at all. That is a diagnostic advantage of this card worth keeping: it removes one variable from every grounding bug you file.

Out of memory at load

Unlikely on this card at the documented configuration — 12.668 GiB accountable against 24 GiB — but in order of what to try if something else is holding VRAM. Drop -c from 131072 to 65536, which frees exactly 2.000 GiB and is still four times the harness's 15,000-token floor; then 32768 for another 1.000 GiB. If that is not enough, the next step is the quant ladder, and from BF16 the first rung is large: Fara1.5-4B-Q8_0.gguf frees 3.661 GiB, Fara1.5-4B-Q6_K.gguf 4.421 GiB, Fara1.5-4B-Q5_K_M.gguf 4.758 GiB. Keep the f16 projector: at 0.626 GiB it is not where the savings are, and the q8_0 projector that one quantiser ships trades vision fidelity in a model whose entire job is looking at pixels. If you need the projector out of VRAM entirely, --no-mmproj-offload keeps it on the CPU at the cost of slower image encoding — once per agent turn. And check what else holds VRAM: on a card that also drives your display, the Chromium this agent opens is competing for the same 24 GB.

Don't reach for speculative decoding to fill the headroom

A card with eleven spare gigabytes invites it, and the model family does carry a multi-token-prediction layer — the source config.json declares mtp_num_hidden_layers: 1 inside text_config. This GGUF does not. Its header has no nextn_predict_layers key (llama.cpp's name for it, per LLM_KV_NEXTN_PREDICT_LAYERS in src/llama-arch.cpp), and none of its 426 tensors is a blk.*.nextn.* tensor: the census stops cleanly at blk.31. The MTP layer was not converted, so --spec-type has nothing to attach to on this file. Nothing on this page enables it; if you go looking, that is why it does nothing.

rope.dimension_sections has wrong array length; expected 4, got 3

This is a real open llama.cpp bug (PR #25334, titled "fix: load qwen35 Ollama GGUF exports"), but it belongs to Ollama-exported qwen35 blobs, not to the GGUF this recipe uses. The BF16 artifact's own header reads qwen35.rope.dimension_sections = [11, 11, 10, 0] — four entries, matching the source config's mrope_section: [11, 11, 10] plus the trailing zero — so it cannot hit this failure. If you see this error, you are loading a different file than the one downloaded in Step 2.

ollama run fara does nothing — there is no Ollama entry

There is no Ollama library entry for Fara1.5 in any form: fara1.5, fara, fara1-5 and fara1.5-4b all 404, and so does the registry manifest API. The GGUF / LM Studio / Ollama line in the harness README refers to Fara-7B, the previous generation, reached with a --fara-7b flag — it is not about Fara1.5. Likewise, the "run them in your choice of tools" list on the quantiser's card is that quantiser's standard boilerplate. llama.cpp built from source is the path this recipe documents.

Generation is slower than a 4B model should be

Two things to separate before you go bug-hunting. The first is expected: BF16 moves 1.875x the bytes per token that Q8_0 does, and that is a cost you chose in exchange for skipping the quantisation question — A/B against Fara1.5-4B-Q8_0.gguf with llama-bench before treating it as a defect. The second is a lead rather than a known property: llama.cpp issue #25162 is an open, community-reported regression against exactly the SSM/linear-attention kernels this architecture leans on, titled "Performance regression on Turing GPUs after 9e58d4d69 — Qwen35 SSM kernels affected". Read its scope carefully: the title says Turing, the reporter measured on an RTX 2080 Ti, the thread's two comments are both from community accounts with no maintainer response, one of them asks whether an unmerged PR already fixes it, and nobody has posted numbers for an Ampere card. It is a thing to check with llama-bench across a couple of builds, not a property of this one.