Fara1.5-9B on RTX 4070: a Local Browser Computer-Use Agent with llama.cpp

What You'll Build

A local computer-use agent on a single RTX 4070: llama-server hosting Fara1.5-9B as a Q5_K_M GGUF plus its vision projector, and Microsoft's fara-cli harness pointed at it, driving a Playwright-launched Chromium at 1440×900. You give it a task in English; it takes a screenshot, reasons, and emits a click, a keystroke or a URL visit, then loops.

Hardware data: RTX 4070 (12GB VRAM) · Q5_K_M + f16 mmproj + 32K context = 8.29 GiB of accountable allocations · See benchmark data

⚠️ This model clicks, types and submits in a real browser, and the default path here is not a sandbox. That the model acts is the product, not a side effect. The model card's Responsible AI section asks you to "run Fara in an isolated container with no access to host files, environment variables, or sensitive credentials", and its Recommended Deployment section states plainly: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." fara-cli launches its own Chromium through Playwright with a fresh browser context — it does not attach to your logged-in browser — but it runs uncontained, on your machine, with your network. Isolation is a separate step you have to take: see Sandbox it below, and do it before pointing this at anything that matters.

ℹ️ Not a general-purpose vision chatbot. Fara1.5-9B is fine-tuned for one loop: screenshot in, <tool_call> out. It is in our multimodal vertical because it takes images and text and emits text, but it is trained against a specific system prompt and a fixed computer_use action schema. Used as a plain image-captioning model it will still answer, but nothing about it is tuned for that. Per the model card: "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree."

Requirements

Component	Minimum	Tested
GPU	12GB VRAM (Q5_K_M + mmproj + 32K context = 8.29 GiB accountable)	not measured — contribute a run
RAM	16GB (model is mmap'd; browser + Playwright on top)	—
Storage	7.24 GiB (Q5_K_M 6.382 GiB + f16 mmproj 0.855 GiB)	~8 GiB with Chromium
Software	llama.cpp with CUDA, Python 3.10+, Playwright	—

Why the 9B, and not the 4B or the 27B

The 9B is the middle of three sizes and has to earn the tier. Microsoft answers this directly in the Fara1.5 paper: "Fara1.5-9B already covers a substantial fraction of the 4B-to-27B gain on both metrics, which makes it the natural choice when deployment cost is a constraint." — followed immediately by "Fara1.5-27B is the right choice when raw accuracy matters more than cost." On a 12 GB card, cost is the constraint, so this is the tier.

The numbers behind that, from the paper's Table 7 (averaged over three independent runs, with 95% confidence intervals — a better source than the bare figures on the model card):

Model	WebVoyager	Online-Mind2Web	WebTailBench v1.5
Fara1.5-4B	80.8 ± 2.3	57.3 ± 4.3	27.4 ± 2.8
Fara1.5-9B	86.6 ± 2.0	63.4 ± 4.3	32.3 ± 2.8
Fara1.5-27B	89.3 ± 1.8	72.3 ± 3.8	40.2 ± 3.0

Microsoft's own reading of those intervals is that they are "small relative to" the gaps between model sizes, confirming the scaling trend is not run-to-run noise — and across the family as a whole that is fair. Doing the arithmetic pairwise gives a more granular picture, and this next part is our reading, not theirs: on WebVoyager the 9B reaches 88.6 at the top of its interval while the 27B drops to 87.5 at the bottom of its own, so the two overlap; on Online-Mind2Web (67.7 vs 68.5) and WebTailBench (35.1 vs 37.2) they do not. Overlapping 95% intervals is only a conservative proxy for a significance test and does not by itself establish that two results are indistinguishable — so read this as the WebVoyager gap is much the weakest-separated of the three, not as a null result. The 27B's advantage is real but not uniform, and it is smallest exactly where the benchmark is easiest.

The paper is specific about where that gap comes from, and it is not speed. Larger models "solve more of the hard tail", and on success rate versus trajectory length: "The mapping is largely independent of size in the bulk of the step range; larger models retain a higher success rate only in the long tail." Nor is the 9B less efficient per task it completes — "The larger mean step counts of smaller models above are largely a consequence of their higher failure rate, not lower efficiency." Expect the 9B to handle ordinary tasks about as directly as the 27B would, and to lose ground specifically on long, hard trajectories. And the ceiling is low for everyone on the hardest benchmark — the paper's own words are that "even the 27B model leaves a substantial fraction of examples unsolved in three attempts" there.

This exact question was asked on the 9B's own HuggingFace discussions in July 2026 — "Does anyone have any suggestions on the kinds of tasks where the 9B model fails where the 27B model appears to be necessary?" — and has no community answer beyond the paper. Nobody has published a qualitative rule; the interval arithmetic above is as sharp as the public evidence gets.

What the tier costs on this card is unusually easy to state, because the 4B and 9B are architecturally identical everywhere except width. Both declare num_key_value_heads: 4, head_dim: 256, 32 blocks and full_attention_interval: 4 in config.json — so the per-token KV cost of the 9B is exactly the same as the 4B's, and so is its fixed recurrent state. The entire difference is weights: 3.088 GiB → 6.382 GiB at Q5_K_M, plus 0.626 → 0.855 GiB of projector, from the 4B and 9B GGUF trees. You pay ~3.5 GiB of otherwise-idle VRAM for those points and nothing at all in running cost.

The 27B is simply out of reach here: its Q4_K_M is 16.329 GiB before the 0.864 GiB projector.

How it fits in 12 GB — the arithmetic

There is no benchmark for this pair yet (/check reports unknown), so every figure below is derived from the artifact and from llama.cpp's own source. Reproduce it rather than trusting it.

Weights. From the bartowski GGUF tree: Fara1.5-9B-Q5_K_M.gguf = 6,852,928,640 B = 6.382 GiB (HuggingFace displays this as 6.85 GB — decimal), mmproj-Fara1.5-9B-f16.gguf = 918,166,048 B = 0.855 GiB.

KV cache — smaller than you expect, because this is a hybrid model. config.json carries a layer_types array of 24 linear_attention + 8 full_attention entries, and llama.cpp agrees: llm_arch_is_hybrid(LLM_ARCH_QWEN35) returns true in src/llama-arch.cpp, and src/models/qwen35.cpp assigns layers with is_recr_impl[i] = (i + 1) % full_attn_interval != 0. With full_attention_interval = 4 and 32 blocks that is layers 3, 7, … 31 — 8 attention layers. Only those get a growing cache:

K+V per token per attention layer = 2 × n_head_kv(4) × head_dim(256) × 2 B (f16) = 4,096 B
× 8 attention layers                                                            = 32 KiB / token

n_ctx = 16,384 → 0.500 GiB
n_ctx = 32,768 → 1.000 GiB
n_ctx = 65,536 → 2.000 GiB

A non-hybrid 32-layer model with the same head geometry would cost four times that.

Recurrent state — fixed, and tiny. The other 24 layers hold a context-independent state. Using llama.cpp's n_embd_r() / n_embd_s() in src/llama-hparams.cpp with this file's own GGUF keys (ssm.conv_kernel 4, ssm.inner_size 4096, ssm.group_count 16, ssm.state_size 128):

conv state : (4−1) × (4096 + 2×16×128) =  24,576 elems
ssm  state :        128 × 4096          = 524,288 elems
per layer  : 548,864 × 4 B (F32)        = 2.094 MiB
× 24 layers                             = 50.25 MiB      ← does not grow with context

Total at the recommended settings:

	GiB
Q5_K_M weights	6.382
f16 mmproj	0.855
KV cache @ 32,768	1.000
Recurrent state	0.049
Accountable total	8.286

That leaves roughly 3.5 GiB on a 12 GB card for compute buffers, the vision encode graph and your desktop — which is why this recipe leads with Q5_K_M rather than Q4_K_M. The headroom is better spent on the quant than on context, and that is a decision with a reason: the harness's documented context floor is 15,000 tokens and it retains only three screenshots, so context beyond ~32K is never used, while the model's job is emitting pixel coordinates. If you want the arithmetic for other tiers: Q4_K_M @ 32K = 7.409 GiB, Q6_K @ 32K = 9.076 GiB.

What a screenshot actually costs. The harness fixes the viewport at 1440×900 (Fara15AgentConfig.viewport_width/height in src/fara/agents/fara/fara15_agent.py) and pre-resizes with Qwen's smart_resize at factor = patch_size × merge_size = 32 → 1440×896. llama.cpp then applies the identical algorithm (calc_size_preserved_ratio in tools/mtmd/mtmd-image.cpp) and clamps to 8–4096 merged tokens for this projector (set_limit_image_tokens(8, 4096) in tools/mtmd/clip.cpp); 1,290,240 px sits inside that, so nothing is rescaled:

(1440 / 32) × (896 / 32) = 45 × 28 = 1,260 tokens per screenshot
× max_n_images = 3                 = 3,780 tokens resident

Microsoft's own harness budgets image_token_estimate: 1500 per image, so 1,260 is the real figure and 1,500 is their conservative rounding. This is why the README asks for at least 15,000 tokens of context and why 32,768 is comfortable.

Installation

1. Build llama.cpp with CUDA

The community GGUF was quantized on release b10087; build that tag or newer.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

Support for this model is first-class in mainline, at both halves — worth confirming before you download 7 GiB:

grep -n 'LLM_ARCH_QWEN35,' src/llama-arch.cpp                      # -> { LLM_ARCH_QWEN35, "qwen35" }
grep -n 'qwen3vl_merger' tools/mtmd/clip-impl.h                    # -> PROJECTOR_TYPE_QWEN3VL
grep -n 'Qwen3_5ForConditionalGeneration' conversion/qwen3vl.py    # -> Qwen3VLVisionModel (the mmproj converter)

2. Download the weights and the projector

There is no first-party GGUF — Microsoft ships safetensors only. Use a community quantization; at least four independent quantizers publish a matching mmproj for the 9B, which is itself corroboration that the vision conversion works.

pip install -U huggingface_hub
hf download bartowski/Fara1.5-9B-GGUF Fara1.5-9B-Q5_K_M.gguf --local-dir ./fara-gguf
hf download bartowski/Fara1.5-9B-GGUF mmproj-Fara1.5-9B-f16.gguf --local-dir ./fara-gguf

Downloading the projector is not optional. In GGUF the vision tower lives in a separate file loaded by tools/mtmd; without it you have a text-only model and a computer-use agent that cannot see. You can verify the artifact you just downloaded declares the projector llama.cpp implements:

strings ./fara-gguf/mmproj-Fara1.5-9B-f16.gguf | grep -m1 qwen3vl_merger

3. Install the fara harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Omit the [vllm] extra from the README's command — that path is for bf16 serving on datacenter cards and is not what this recipe uses.

4. Point the harness at your local server

cat > local_llamacpp.json <<'EOF'
{
    "model": "Fara1.5-9B",
    "base_url": "http://127.0.0.1:5000/v1",
    "api_key": "sk-no-key-required"
}
EOF

Running

Start the server

./build/bin/llama-server \
  -m ./fara-gguf/Fara1.5-9B-Q5_K_M.gguf \
  --mmproj ./fara-gguf/mmproj-Fara1.5-9B-f16.gguf \
  --reasoning-format none \
  --image-min-tokens 1024 \
  -c 32768 \
  -ngl 99 \
  --temp 0 \
  --host 127.0.0.1 --port 5000

Three of those flags are load-bearing and none of them is a default:

• --reasoning-format none — mandatory. See Troubleshooting; without it the agent loop silently receives nothing.
• --image-min-tokens 1024 — llama.cpp's default floor for this projector is 8 merged tokens, and tools/mtmd/clip.cpp warns that "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks". A full 1440×900 screenshot is 1,260 tokens and clears the floor on its own, but any smaller image the agent sends would not.
• -c 32768 — the harness README requires "Please ensure that context length is set to at least 15000 tokens and temperature to 0 for best results."

If you would rather not manage files, -hf bartowski/Fara1.5-9B-GGUF:Q5_K_M fetches the model and, per llama.cpp's server docs, the matching mmproj alongside it.

Run a task

source .venv/bin/activate
fara-cli \
  --task "find the current UTC time on time.is and tell me" \
  --endpoint_config local_llamacpp.json \
  --output_folder ./run1 \
  --save_screenshots \
  --max_rounds 15

The browser is headless by default; add --headful to watch it work, which you should do the first few times. Each step writes a screenshot and a data_point.json trajectory into --output_folder. The model stops on its own at critical points — entering personal data, payments, submissions, sign-ins — and asks rather than proceeding.

Sandbox it

fara-cli gives you a fresh Chromium context, not container isolation. For anything beyond a first smoke test, run the harness inside a container with no host mounts and no credentials in its environment, or use Magentic-UI, which the model card names as the recommended deployment and which provides Docker sandboxing, domain allow-lists, watch-mode and a pause control. Match the sandbox viewport to the trained resolution — the card is explicit: "Match this in your sandbox for the most reliable grounding."

Results

• Speed: omitted. No benchmark of Fara1.5-9B on an RTX 4070 — or any consumer GPU — exists at the time of writing, and Microsoft publishes no throughput figures at all — the card names A6000, A100, H100 and B200 only as hardware its bf16 serving path was tested on, which says nothing about a Q5_K_M build on Ada. If you measure it, please contribute the numbers so this line can be filled in from real data rather than extrapolation.
• VRAM usage: 8.29 GiB of accountable allocations at Q5_K_M / 32K context, derived above from file sizes and llama.cpp's own cache formulas. Compute buffers sit on top of that and are not derivable from static inspection — expect to confirm the true peak with nvidia-smi on your first run. Track the empirical picture at /check/fara1-5-9b/rtx-4070.
• Quality notes: the vision path here is structurally proven, not runtime-proven. Four independent facts establish that it loads: mainline llama.cpp registers Qwen3_5ForConditionalGeneration for both the text model (conversion/qwen.py) and the vision tower (conversion/qwen3vl.py); the shipped mmproj declares clip.projector_type = qwen3vl_merger, which maps to PROJECTOR_TYPE_QWEN3VL in tools/mtmd/clip-impl.h; four quantizers converted it independently; and llama-server documents image_url content parts, which is exactly what the harness sends. What does not exist is a published report of Fara1.5 captioning an image or grounding a click under llama-server. The closest public datapoint is a Fara1.5-27B user running the model under llama-server inside an agent harness (below) — the family and runtime are confirmed working end-to-end; a first-party demonstration of the mmproj decode for Fara1.5 specifically is not.

Troubleshooting

The agent does nothing — every step logs an empty response or list index out of range

This is the single failure everyone hits, and it is not a bug in your setup. Fara opens a <think> block and never closes it. Under llama-server's default reasoning extraction the parser therefore treats the whole output — reasoning and the <tool_call> block together — as reasoning, returns it in reasoning_content, and leaves message.content empty. The harness parses message.content for <tool_call>, finds nothing, and the loop dies.

Launch with --reasoning-format none, which llama.cpp's server documentation describes as leaving thoughts unparsed in message.content. The raw output then begins with a stray unclosed <think> tag, which is cosmetic.

Reported for Fara1.5-27B by a community user on llama.cpp b10107 in the model's HF discussions and acknowledged there by Microsoft — swhitehead (a Fara org member) replied "Would you mind posting this as an issue on the Fara1.5 github repo so we can mention this in the hosting instructions?", which became microsoft/fara#82. It applies to the 9B for a checkable reason rather than by assumption: the 9B's own GGUF carries the same chat template, whose generation prompt emits <think>\n with no closing tag whenever enable_thinking is not explicitly false. The same reporter adds: "If you have a router/proxy in front doing its own reasoning extraction, disable it there too."

Clicks land in the wrong place

Two things to check before blaming the model. First, the viewport must be 1440×900 — the harness's default, and the resolution the card says to match. Second, confirm --image-min-tokens 1024 is set; llama.cpp's default floor for this projector is 8 merged tokens, and grounding is exactly what that warning is about.

llama-server starts but images are ignored

If you launched with -m and no --mmproj, you have loaded a text-only model — the automatic projector download only happens on the -hf path. Check the startup log for the clip/mmproj load lines, and check /v1/models for the multimodal capability, which llama.cpp's server docs tell clients to test before sending images.

Out of memory on a 12 GB card

In order of what to try: drop -c from 32768 to 16384 (frees 0.5 GiB), move down to Q4_K_M (frees a further 0.877 GiB), or pass --no-mmproj-offload to keep the projector on the CPU (frees 0.855 GiB at the cost of slower image encoding). Note the third option only slows the vision step, which happens once per agent turn.

Running it against your own accounts

Don't. The model is trained to pause before sign-ins, payments and submissions, and that training is a safety net, not a guarantee — the card lists prompt injection from page content among its known risks, and a vision-only agent has no way to distinguish a legitimate button from a hostile one that renders like it. Keep it in a container, on an allow-list, with no credentials in reach.

For the full benchmark data, see /check/fara1-5-9b/rtx-4070.