Fara1.5-4B on RTX 4070: a Q8_0 Browser Computer-Use Agent with llama.cpp

What You'll Build

A local browser computer-use agent (CUA): Microsoft's Fara1.5-4B served by llama-server on an RTX 4070, with the vision projector loaded so the model actually sees browser screenshots, driven by Microsoft's own fara-cli harness against a plain local Playwright Chromium.

The 12 GB on this card changes what the recipe should be. An 8 GB card has to take a 4-bit quant and a short context; here the same model fits at Q8_0 — the closest tier to the weights Microsoft actually evaluated — and still has room for a 65,536-token context. Both halves of that trade are derived below.

Hardware data: RTX 4070 (12GB VRAM) · Q8_0 + f16 mmproj = 4.812 GiB of weights, 7.008 GiB accountable at 65,536 context (derived below) · See benchmark data

⚠️ This model clicks, types and submits in a real browser, and the path documented here gives it no isolation. That is the operating requirement, not boilerplate, and it is the thing to settle before anything else on this page.

The model card is blunt about the consequence: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." That sentence closes the card's Recommended Deployment: MagenticLite section, which lists sandboxing, allow-lists, watch-mode and a pause control as MagenticLite features and then says: "If you integrate Fara1.5-4B directly, you're responsible for these controls."

fara-cli is a direct integration. Its install step is playwright install, and it drives a plain local Playwright Chromium — no container, no allow-list, no pause control. The harness README offers the containerised option as an alternative to it, not as part of it: "instead of fara-cli you can use Magentic-UI", described there as "a sandboxed browser environment with auditable action logging and user prompts at critical points". For the CLI path the vendor's position is a recommendation, not something shipped: "We recommend running it in a sandboxed environment, monitoring its execution, and avoiding sensitive data or high-risk domains."

So on the fara-cli path you are the isolation boundary. Practically: a throwaway OS user or a VM, Playwright's own fresh browser profile (never your daily one), signed out of everything, low-stakes public sites, and eyes on the run rather than leaving it unattended. If you want the container, the allow-lists, watch-mode and the pause control, use Magentic-UI as the client instead — see Running.

ℹ️ Vision is the product, not a bonus. Per the model card, "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree." A Fara install that only serves text is a broken install — the --mmproj step below is mandatory, and Step 4 exists to prove that an image really reached the model before you let it drive anything.

⚠️ Runtime status — read before you commit an evening to this. llama.cpp support for this architecture is verifiable statically, and this recipe shows that evidence rather than asserting it. What does not exist anywhere public is a report of Fara1.5 decoding an image under llama-server or llama-mtmd-cli. The closest datapoint is a community user serving Fara1.5-27B under llama-server on build b10107 and driving it from an agent harness successfully — but that thread is about tool-call parsing, not about the projector. Treat your first run as the test, and see Results for why there is no speed figure here either.

Requirements

Component	Minimum	This recipe
GPU	12GB VRAM, CUDA compute 8.9 (Ada)	not measured — contribute a run
RAM	16GB (weights are mmap'd; Chromium sits on top)	—
Storage	5.17 GB (Q8_0 + f16 mmproj)	5,166,377,728 B = 4.812 GiB
Software	llama.cpp b10087+ (CUDA), Python 3.10+, Playwright	—

Microsoft ships safetensors only — the two shards total 8.455 GiB per the HF tree API (5,960,243,904 + 3,118,376,552 B). Its Quickstart serves those with vLLM and names datacenter hardware for the job: "A GPU with enough memory for a 4B model in bf16 (A6000, A100, H100, and B200 have been tested)", and the card's highlights claim only that the model "runs on a single A100/H100/B200 with room for the screenshot history". The consumer route is the quantised one documented here.

What 12 GB actually buys

There is no benchmark for this pair yet — /check reports unknown — so every number below is arithmetic from the artifact's own metadata and llama.cpp's source, not a measurement. Reproduce it rather than trusting it.

Weights. Byte counts from the bartowski GGUF tree:

Component	Bytes	GiB
Fara1.5-4B-Q8_0.gguf	4,493,954,144	4.185
mmproj-Fara1.5-4B-f16.gguf	672,423,584	0.626
Weights total	5,166,377,728	4.812

KV cache — and this is where Fara1.5 is unusual. Its qwen35 architecture is a hybrid: llm_arch_is_hybrid() in src/llama-arch.cpp returns true for LLM_ARCH_QWEN35, and src/models/qwen35.cpp marks layers recurrent with is_recr_impl[i] = (i < hparams.n_layer()) && ((i + 1) % full_attn_interval != 0). The Q8_0 file's own header carries qwen35.block_count = 32 and qwen35.full_attention_interval = 4, and the source config.json spells the split out layer by layer: 24 of 32 layers are linear_attention, only 8 are full_attention. Only those 8 carry a cache that grows with context.

With qwen35.attention.head_count_kv = 4 and attention.key_length = attention.value_length = 256:

K+V per token per attention layer = 2 x 4 x 256 x 2 B (f16) = 4,096 B
x 8 full-attention layers                                   = 32 KiB / token

A non-hybrid 32-layer model with the same head geometry would cost four times that. Context on this model is cheap, and that is the fact the 12 GB budget turns on.

Recurrent state — fixed, and tiny. The other 24 layers hold a context-independent state, sized by n_embd_r() / n_embd_s() in src/llama-hparams.cpp from the artifact's own GGUF keys (ssm.conv_kernel 4, ssm.inner_size 4096, ssm.group_count 16, ssm.state_size 128):

conv state : (4-1) x (4096 + 2 x 16 x 128) =  24,576 elems
ssm  state :         128 x 4096            = 524,288 elems
per layer  : 548,864 x 4 B (F32)           = 2.094 MiB
x 24 layers                                = 50.25 MiB per sequence
x 4 sequences                              = 201 MiB = 0.196 GiB   <- does not grow with context

That last multiplication is easy to miss and it is not optional. src/llama-model.cpp sizes the recurrent state with /* recurrent_rs_size */ std::max((uint32_t) 1, cparams.n_seq_max), and tools/server/server.cpp resolves the server's auto default with params.n_parallel = 4, so a plain llama-server reserves four sequences' worth whether or not you use them. The KV cache is not multiplied the same way: that same default also sets params.kv_unified = true, so the 32 KiB/token derived above is the whole KV cost regardless of slot count.

The trade, in one table. Accountable allocations (weights + projector + KV + recurrent state) in GiB, each cell summed from exact byte counts and rounded once:

Context	Q4_K_M	Q5_K_M	Q6_K	Q8_0
32,768	4.509	4.910	5.247	6.008
65,536	5.509	5.910	6.247	7.008
131,072	7.509	7.910	8.247	9.008
262,144	11.509	11.910	12.247	13.008

This recipe leads Q8_0 at 65,536, and the reasoning is worth stating because the cheap option is not the obviously wrong one:

• The quant tier is where the risk is. Fara's output is pixel coordinates — it is a grounding model, and the vendor's published scores are for bf16 weights. No public evaluation measures how much grounding accuracy any quantisation costs. Q8_0 is the tier closest to what was evaluated, and buying it here costs 1.499 GiB over Q4_K_M on a card that has the room. On 8 GB that trade is not available; on 12 GB, declining it is a choice you would have to defend.
• Context is worth buying second, not first. At 32 KiB/token, doubling the 8 GB tier's 32,768 to 65,536 costs exactly 1.000 GiB — cheap, and it is 4.3x the harness's documented floor (see below). Going further to 131,072 still fits at 9.008 GiB if your trajectories are long.
• The model's native 262,144 does not fit at any tier. Even Q4_K_M lands at 11.509 GiB of accountable allocations before a single compute buffer, on a 12 GB card. Cap -c.

At Q8_0 / 65,536 the 7.008 GiB above leaves roughly 5 GiB of the card for the CUDA context, compute buffers, the vision encode graph and your desktop. Those terms are not in the table — they are allocated at load time and this recipe does not derive them. Read the CUDA0 compute buffer size lines llama-server prints on startup for the real figures on your build, and report what you see.

Why not just run bf16? Because it barely fits and buys nothing you can point at. The bf16 GGUF is 7.846 GiB and its bf16 projector 0.629 GiB — 8.475 GiB of weights, which forces the context back down to 32,768 (9.671 GiB accountable) and leaves ~2.3 GiB for everything else. Q8_0 is the honest ceiling on this card.

How the context is actually spent

A CUA sends a screenshot every step, so the per-image cost decides whether the budget above is realistic. It is not the 768 in the projector metadata: that value derives from the vision tower's position-embedding grid and llama.cpp treats this projector as dynamic-resolution — --image-min-tokens and --image-max-tokens are documented as "only used by vision models with dynamic resolution".

The harness fixes the browser viewport at 1440x900 (viewport_width: int = 1440, viewport_height: int = 900 in fara15_agent.py), matching the card's note that "The screen resolution Fara is most commonly trained with is" 1440x900. It pre-resizes with Qwen's smart_resize at PATCH_SIZE = 16 and MERGE_SIZE = 2, so the factor is 32 and 900 becomes 896. llama.cpp then computes patches the same way — x_patch = nx / (patch_size * 2), y_patch = ny / (patch_size * 2) in tools/mtmd/clip.cpp:

(1440 / 32) x (896 / 32) = 45 x 28 = 1,260 tokens per screenshot
x max_n_images = 3                 = 3,780 tokens resident

The harness's own conservative default is image_token_estimate: int = 1500, which is a useful sanity check on the arithmetic, and the model card agrees on the retention: "We only keep the most recent 3 screenshots" in the chat history.

So images are capped. What grows is the text trajectory: every round appends the model's reasoning, its tool call and an observation (max_observation_chars: int = 1000), and fara-cli defaults to --max_rounds 100. That is why the extra context is not a spec-sheet number — a long run accumulates history that a 15,000-token floor would truncate. The harness README asks for that floor explicitly: "Please ensure that context length is set to at least 15000 tokens and temperature to 0 for best results." Microsoft's own framing of the model's 262K window is "Long enough for multi-screenshot trajectories with full action history."

Installation

1. Build llama.cpp with CUDA

Ada Lovelace is sm_89. Build from a release at or after b10087 — the release this GGUF was quantized on.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

Both halves of the model are first-class in mainline — worth confirming before you download 5 GB:

grep -n 'LLM_ARCH_QWEN35,' src/llama-arch.cpp        # -> { LLM_ARCH_QWEN35, "qwen35" }
grep -n 'qwen3vl_merger' tools/mtmd/clip-impl.h      # -> PROJECTOR_TYPE_QWEN3VL

2. Download the weights and the projector

Both files are required. The projector is a separate artifact — downloading only the text GGUF gives you a model that cannot see.

pip install -U huggingface_hub

hf download bartowski/Fara1.5-4B-GGUF \
  Fara1.5-4B-Q8_0.gguf mmproj-Fara1.5-4B-f16.gguf \
  --local-dir ~/models/fara1.5-4b

Microsoft publishes no first-party GGUF for Fara1.5 — the microsoft org ships safetensors for all three sizes, plus an ONNX repo for the previous-generation Fara-7B, which is a different model and should never be mixed in. Four independent community quantisers have converted the 4B including its vision tower: bartowski, prithivMLmods, DevQuasar and runanywhere each publish a matching mmproj. Four independent conversions succeeding is the strongest available evidence that the vision path converts cleanly; it is still not a runtime test.

3. Serve it

./build/bin/llama-server \
  -m ~/models/fara1.5-4b/Fara1.5-4B-Q8_0.gguf \
  --mmproj ~/models/fara1.5-4b/mmproj-Fara1.5-4B-f16.gguf \
  -ngl 99 -c 65536 --host 127.0.0.1 --port 5000 \
  --image-min-tokens 1024 \
  --reasoning-format none \
  --temp 0

Three of those flags are load-bearing and none of them is a default:

• --reasoning-format none — mandatory. Without it the agent loop silently receives nothing; see Troubleshooting for the mechanism.
• --image-min-tokens 1024 — this projector ships no minimum-pixel key, so llama.cpp falls back to a floor of 8 merged tokens and warns: "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks". Fara is a grounding model, so that warning is aimed precisely at this case. At the harness's viewport the image is already 1,260 tokens and the floor never binds; it protects you if you shrink the viewport.
• -c 65536 — see the budget table. Do not leave it at the model's native 262,144.

If you would rather not manage files, -hf bartowski/Fara1.5-4B-GGUF:Q8_0 fetches the model, and per the server docs "mmproj is also downloaded automatically if available".

4. Prove the vision path works before going further

Check that the server advertises multimodal support, then send a real image. The server docs make the first part a client-side requirement rather than an optional nicety: "Clients should check /models or /v1/models for the multimodal capability before a multimodal request." Run GET /props alongside it — it reports which modalities loaded rather than merely that some did, which is the more useful signal when a projector silently failed to load. For the image itself, image_url.url "can be a remote URL, base64 (raw or URI-encoded via data:image/...;base64) or path to local file".

curl -s http://127.0.0.1:5000/v1/models | grep -o multimodal

# supplementary — names the modality that actually loaded
curl -s http://127.0.0.1:5000/props | python3 -c "import sys,json; print(json.load(sys.stdin)['modalities'])"
# expect: {'vision': True}

curl -s http://127.0.0.1:5000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "fara",
  "temperature": 0,
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "Describe this image in one sentence."},
    {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}
  ]}]
}' | python3 -m json.tool

If the reply describes the picture, the projector is wired up. If /v1/models never prints multimodal, if /props reports vision: False, or if the model answers as though no image were present, stop here — the agent loop below will produce confidently wrong actions rather than fail loudly.

5. Install the harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Skip the README's [vllm] extra — that path serves bf16 on datacenter cards and is not what this recipe uses.

Running

Two clients share the same llama-server endpoint. Path A (fara-cli) is the smoke test, with no isolation. Path B (Magentic-UI) is what the vendor recommends for anything real. They are not equivalent in risk; pick deliberately.

Path A — fara-cli, for a first smoke test

fara-cli speaks to any OpenAI-compatible endpoint. Its built-in default already points at http://localhost:5000/v1, but name the model explicitly since the default config names a different size:

source .venv/bin/activate
fara-cli \
  --base_url http://127.0.0.1:5000/v1 \
  --api_key sk-no-key-required \
  --model fara \
  --task "find the current UTC time on time.is and tell me" \
  --output_folder ./runs/first \
  --save_screenshots \
  --max_rounds 15

The browser runs headless by default; add --headful to watch it work, which you should do the first few times. Each step writes a screenshot and a data_point.json trajectory into --output_folder — watch that folder early, because it is how you tell a grounding failure (clicks landing in the wrong place) from a reasoning failure.

Keep the task on a low-stakes public site and stay signed out. Fara is trained to stop and ask before entering personal information, making payments, submitting forms, signing in or sending messages; if you see it sail through one of those, stop the run — that is a wiring problem in your prompt path, not a quirk to tolerate.

Path B — Magentic-UI, for the container

Everything the model card assumes — the containerised browser, domain allow-lists, watch-mode and a pause control — lives in Magentic-UI, not in fara-cli. That is also where a Docker requirement would come from; there is no Docker step in Path A because Path A genuinely does not use one.

Magentic-UI takes the same OpenAI-compatible endpoint you started above, so nothing on this page changes except the client. Its setup is documented in its own repository rather than reproduced here, because this recipe has not verified those steps end to end on this GPU — give it http://127.0.0.1:5000/v1 as the model endpoint. Whichever client you use, match the trained resolution: the card is explicit — "Match this in your sandbox for the most reliable grounding."

Results

• Speed: omitted — no published measurement of Fara1.5-4B exists for this card or for any consumer GPU, and the only hardware Microsoft names is datacenter class. Borrowing a figure from an A100 for a 12 GB Ada card would be misleading, and the vendor publishes no throughput numbers to borrow in the first place. If you run this, please contribute your numbers so this line can be replaced with a measurement.
• VRAM usage: 4.812 GiB of weights; 7.008 GiB accountable including KV and the four-sequence recurrent state at 65,536 context, derived above from the GGUF's own metadata and llama.cpp's memory formulas. This is a derivation, not an observed peak — compute and vision buffers sit on top. Track the empirical picture at /check/fara1-5-4b/rtx-4070.
• Quality notes: the vendor's published scores (80.8 WebVoyager, 57.3 Online-Mind2Web, 27.4 WebTailBench) are for the bf16 model on datacenter hardware, and no published evaluation measures what quantisation costs a model whose job is predicting exact pixel coordinates. Q8_0 is the closest tier to those weights, which is why this recipe spends the card's headroom there rather than on a context nothing in the harness will fill.
• Language: English only — the card puts "Languages other than English (training data is English-only)" out of scope.

For the full benchmark data, see /check/fara1-5-4b/rtx-4070.

Troubleshooting

The agent does nothing — every step logs an empty response or list index out of range

This is the failure everyone hits, and it is not a bug in your setup. Fara opens a <think> block and never closes it. Under llama-server's default reasoning extraction the parser therefore treats the whole output — reasoning and the <tool_call> block together — as reasoning, returns it in reasoning_content, and leaves message.content empty. The harness parses message.content for <tool_call>, finds nothing, and the loop dies on round one.

Launch with --reasoning-format none, documented as "none: leaves thoughts unparsed in message.content". The raw output then begins with a stray unclosed <think> tag, which is cosmetic.

Reported for Fara1.5-27B by a community user on build b10107 in the model's HF discussions and acknowledged there by Microsoft — swhitehead (an org member) replied "Would you mind posting this as an issue on the Fara1.5 github repo so we can mention this in the hosting instructions?", which became microsoft/fara#82, still open. It applies to the 4B for a checkable reason rather than by assumption: the 4B's own GGUF carries the same chat template, whose generation prompt emits an unclosed <think>. The same reporter adds: "If you have a router/proxy in front doing its own reasoning extraction, disable it there too."

The model answers but ignores the screenshot

Almost always the projector. Confirm --mmproj is on the command line, that the server logged an mmproj load stage, that /v1/models advertises the multimodal capability, and that Step 4's /props call reports vision: True. A text-only GGUF with no projector loads and runs perfectly happily — it simply cannot see, which in a CUA presents as confidently wrong actions rather than an error.

Clicks land in the wrong place

Check the viewport first: the harness defaults to 1440x900 because that is what the model was mostly trained at, and a different window size changes both the grounding distribution and the token cost derived above. Then confirm --image-min-tokens 1024 is set, since llama.cpp's own warning ties sub-1024-token images to degraded grounding. Only after both should you suspect the weights — and at Q8_0 you are already at the top of the ladder, so the answer is more likely the viewport than the quant.

rope.dimension_sections has wrong array length; expected 4, got 3

This is a real open llama.cpp bug (PR #25334), but it belongs to Ollama-exported qwen35 blobs, not to the GGUF this recipe uses. The Q8_0 artifact's own header reads qwen35.rope.dimension_sections = [11, 11, 10, 0] — four entries — so it cannot hit this failure. If you see this error, you are loading a different file than the one downloaded in Step 2.

ollama run fara does nothing — there is no Ollama entry

There is no Ollama library entry for Fara1.5 in any form: fara1.5, fara, fara1-5 and fara1.5-4b all 404, and so does the registry manifest API. The GGUF/LM Studio/Ollama line in the harness README refers to Fara-7B, the previous generation, reached with a --fara-7b flag — it is not about Fara1.5. Likewise, the "run them in your choice of tools" list on the quantiser's card is that quantiser's standard boilerplate, printed with its own caveat that a newly supported model "may need to wait for an update from the developers". llama.cpp built from source is the path this recipe documents.

Generation is slower than a 4B model should be

There is an open, community-reported performance regression against exactly the SSM kernels this architecture leans on — llama.cpp issue #25162 reports a 24–42% loss after commit 9e58d4d69, measured by its reporter on an RTX 2080 Ti (Turing, SM75). Two caveats before acting on it: it has no maintainer response, and nobody has posted numbers for an Ada card, so it is a lead to check with llama-bench across a couple of builds rather than a known property of this one. A commenter asks whether PR #25185 resolves it; as of writing the thread has no answer.

Out of memory at load

In order of what to try: drop -c from 65536 to 32768 (frees exactly 1.000 GiB of KV and is still more than twice the harness's 15,000-token floor), then step down one quant tier — Q6_K frees a further 0.761 GiB, Q5_K_M 1.097 GiB. Keep the f16 projector: at 0.626 GiB it is not where the savings are, and the q8_0 projector some quantisers ship trades vision fidelity in a model whose entire job is looking at pixels. If you need the projector out of VRAM entirely, --no-mmproj-offload keeps it on the CPU at the cost of slower image encoding — once per agent turn.