Fara1.5-4B on RTX 3060 Ti: Browser Computer-Use Agent with llama.cpp Vision

What You'll Build

A local browser computer-use agent (CUA): Microsoft's Fara1.5-4B served by llama-server on an RTX 3060 Ti, with the vision projector loaded so the model actually sees browser screenshots, driven by Microsoft's own fara-cli harness against a plain local Playwright Chromium.

Hardware data: RTX 3060 Ti (8GB VRAM) · Q4_K_M + f16 mmproj = 3.313 GiB of weights, 4.362 GiB total at 32K context (derived below) · See benchmark data

⚠️ This model clicks, types and submits in a real browser — and fara-cli does not sandbox it. That is not boilerplate; it is the operating requirement, and it is the one thing to get right before anything else on this page. The model card is blunt: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." Fara also reads whatever is on the page, so a hostile page is a prompt-injection surface.

Be clear about what the tooling gives you. fara-cli drives a plain local Playwright Chromium — no container, no allow-list, no pause control. The sandbox belongs to Magentic-UI, which the harness README describes as "a sandboxed browser environment with auditable action logging and user prompts at critical points" and offers as what you use "instead of fara-cli you can use Magentic-UI". The vendor's position is a recommendation, not a guarantee shipped in the CLI: "We recommend running it in a sandboxed environment, monitoring its execution, and avoiding sensitive data or high-risk domains."

So if you follow the fara-cli path below, you are the sandbox. Practically: run it on a throwaway OS user or VM, let Playwright use its own fresh browser profile (never your daily one), stay logged out of everything, keep tasks on low-stakes public sites, and watch the run rather than leaving it unattended. If you want the container, allow-lists, watch-mode and pause the model card asks for, use Magentic-UI instead — see Running.

ℹ️ Vision is the product, not a bonus. Per the model card, "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree." A Fara install that only serves text is a broken install — the --mmproj step below is mandatory, and Step 4 exists to prove an image really reached the model before you let it drive anything.

⚠️ Runtime status — read before you commit an evening to this. llama.cpp support for this architecture is verifiable statically and this recipe shows that evidence, but at the time of writing no published issue thread, user report, or docs entry shows Fara1.5 specifically captioning an image under llama-server or llama-mtmd-cli. The qwen35 text arch and the qwen3vl_merger projector are both first-class in llama.cpp master, and four independent quantisers have converted this model's vision tower successfully — but that is static evidence, not a tested path. Treat the first run as the test. There is also no published speed measurement for this pair (see Results).

Requirements

Component	Minimum	Tested
GPU	8GB VRAM, CUDA compute 8.6	not measured — contribute a run
RAM	16GB	—
Storage	3.56 GB (Q4_K_M + mmproj)	3.56 GB decimal / 3.313 GiB
Software	llama.cpp b10087+ (CUDA), Python 3.10+, Playwright	—

The BF16 weights total 8.455 GiB across two safetensors shards — 5.551 + 2.904 GiB per the HF tree API, corroborated by model.safetensors.index.json's metadata.total_size of 9,078,531,072 bytes. (The whole repo is 8.483 GiB; the extra 0.028 GiB is tokenizer and config files, not weights.) Either way the unquantised path does not fit this card. Microsoft's own Quickstart targets datacenter hardware — the card's requirements line says "A6000, A100, H100, and B200 have been tested", and its highlights claim only that the model "runs on a single A100/H100/B200 with room for the screenshot history". The consumer route is the quantised one documented here.

The VRAM budget, derived

There is no benchmark for this pair yet, so every number below is arithmetic from the artifact's own metadata rather than a measurement.

Weights — byte counts from the bartowski GGUF tree:

Component	Bytes	GiB
Fara1.5-4B-Q4_K_M.gguf	2,884,850,784	2.687
mmproj-Fara1.5-4B-f16.gguf	672,423,584	0.626
Weights total	3,557,274,368	3.313

KV cache — and this is where Fara1.5 is unusual. Its qwen35 architecture is a hybrid: llama.cpp's llm_arch_is_hybrid() in src/llama-arch.cpp returns true for LLM_ARCH_QWEN35, and llama-memory-hybrid.cpp splits layers with the filter !hparams.is_recr(il). The GGUF reports qwen35.block_count = 32 and qwen35.full_attention_interval = 4, and the source config.json spells the split out layer by layer: 24 of 32 layers are linear_attention, only 8 are full_attention. So only 8 layers carry a KV cache that grows with context.

With qwen35.attention.head_count_kv = 4 and attention.key_length = attention.value_length = 256:

per layer per token = (4 x 256) + (4 x 256)      = 2,048 elements
x 8 full-attention layers                        = 16,384 elements/token
at f16                                           = 32 KiB/token

The other 24 layers hold a fixed-size recurrent state, sized by llama.cpp's n_embd_r() / n_embd_s() in src/llama-hparams.cpp, fed by the GGUF's ssm.conv_kernel = 4, ssm.inner_size = 4096, ssm.group_count = 16, ssm.state_size = 128:

n_embd_r = (4-1) x (4096 + 2 x 16 x 128)  =    24,576
n_embd_s = 128 x 4096                     =   524,288
(24,576 + 524,288) x 24 layers x 4 bytes  = 50.25 MiB, independent of context

Totals on an 8 GB card:

Context	Weights	KV (f16)	Recurrent state	Sum
16,384	3.313 GiB	0.500 GiB	0.049 GiB	3.862 GiB
32,768	3.313 GiB	1.000 GiB	0.049 GiB	4.362 GiB
65,536	3.313 GiB	2.000 GiB	0.049 GiB	5.362 GiB

The model's full 262,144-token context would need 8 GiB of KV alone — cap it. This recipe uses 32,768, roughly double the harness minimum, leaving ~3 GiB of the card for CUDA context, compute buffers and the vision graph. Those last terms are not in the table: they are allocated at load time and this recipe does not derive them. Read the CUDA0 compute buffer size lines that llama-server prints on startup for the real figures on your build, and report what you see.

How much context one screenshot costs

A CUA sends a screenshot every step, so this is the number that decides whether the context budget above is honest. It is not the 768 in the projector metadata — that value is derived from num_position_embeddings (√2304 × 16 = 768) as a reference for position-embedding interpolation, and llama.cpp treats this projector as dynamic-resolution: clip.cpp computes n_patches as (nx / (patch_size*2)) * (ny / (patch_size*2)), with a --image-min-tokens / --image-max-tokens pair documented as "only used by vision models with dynamic resolution".

The harness fixes the browser viewport at 1440×900 (fara15_agent.py: viewport_width: int = 1440, viewport_height: int = 900), matching the card's note that "The screen resolution Fara is most commonly trained with is" 1440×900. It then applies Qwen's smart_resize with patch_size=16, merge_size=2, so the factor is 32:

900 -> round(900/32)*32   = 896        1440 -> round(1440/32)*32 = 1440
patch grid  = (896/16) x (1440/16)     = 56 x 90 = 5,040
image tokens = 5,040 / (2 x 2)         = 1,260 tokens per screenshot

The harness keeps max_n_images: int = 3, so 3,780 tokens of any prompt are screenshots. Its own conservative default image_token_estimate: int = 1500 sits just above the derived 1,260, which is a useful sanity check on the arithmetic. Add the computer_use schema in the system prompt plus accumulated reasoning and action history and the harness's own floor makes sense — the repo asks that you "Please ensure that context length is set to at least 15000 tokens and temperature to 0 for best results."

Installation

1. Build llama.cpp with CUDA

Ampere is sm_86. Build from a release at or after b10087 — that is the release the GGUF used here was quantised on.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

2. Download the weights and the projector

Both files are required. The projector is a separate artifact — downloading only the text GGUF gives you a model that cannot see.

pip install -U huggingface_hub

hf download bartowski/Fara1.5-4B-GGUF \
  Fara1.5-4B-Q4_K_M.gguf mmproj-Fara1.5-4B-f16.gguf \
  --local-dir ~/models/fara1.5-4b

Microsoft publishes no first-party GGUF for Fara1.5 — the microsoft org ships safetensors for all three sizes (and an ONNX repo only for the previous-generation Fara-7B). Four independent community quantisers have converted the 4B including its vision tower — bartowski, prithivMLmods, DevQuasar and runanywhere all ship an mmproj file. Four independent conversions succeeding is the strongest available evidence that the vision path converts cleanly; it is still not a runtime test.

3. Serve it

./build/bin/llama-server \
  -m ~/models/fara1.5-4b/Fara1.5-4B-Q4_K_M.gguf \
  --mmproj ~/models/fara1.5-4b/mmproj-Fara1.5-4B-f16.gguf \
  -ngl 99 -c 32768 --port 5000 \
  --image-min-tokens 1024 \
  --reasoning-format none \
  --temp 0

Three of those flags are load-bearing and worth understanding rather than pasting:

• --image-min-tokens 1024. The projector ships no image_min_pixels key, so llama.cpp falls back to its Qwen-VL default floor of 8 tokens and prints a warning: "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks". Fara is a grounding model — it emits pixel coordinates — so this is precisely the case the warning is about. At the harness's 1440×900 viewport the image is already 1,260 tokens and the floor never binds; it protects you if you shrink the viewport.
• --reasoning-format none. The default is deepseek, which extracts <think> content into a separate reasoning_content field. Fara's chat template opens every assistant turn with <think>, and the harness parses the raw text itself — it splits on <tool_call> and feeds the preceding text back as the agent's thoughts. none is documented as "none: leaves thoughts unparsed in message.content", which is exactly what the harness expects.
• -c 32768. See the budget table. Do not leave it at the model's native 262,144.

4. Prove the vision path works before going further

Check the server advertises multimodal, then send a real image. The server README documents image_url as a first-class content part: "can be a remote URL, base64 (raw or URI-encoded via" a data URI "or path to local file".

curl -s http://localhost:5000/v1/models | grep -o multimodal

curl -s http://localhost:5000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "fara",
  "temperature": 0,
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "Describe this image in one sentence."},
    {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}
  ]}]
}' | python3 -m json.tool

If the reply describes the picture, the projector is wired up. If it answers as though no image were present, or the server never logged an mmproj load, stop here — the agent loop below will silently produce nonsense actions rather than fail loudly.

5. Install the harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Running

Two client paths share the same llama-server endpoint. Path A (fara-cli) is the smoke test — unsandboxed. Path B (Magentic-UI) is what the vendor recommends for anything real. Pick deliberately; they are not equivalent in risk.

Path A — fara-cli, for a first smoke test

fara-cli speaks to any OpenAI-compatible endpoint. Point it at llama-server:

fara-cli \
  --base_url http://localhost:5000/v1 \
  --api_key sk-no-key-required \
  --model fara \
  --task "how many pages does wikipedia have" \
  --output_folder ./runs/first

Each run writes per-step screenshots and a data_point.json recording the task, actions, observations and outcome to --output_folder. Watch that folder on the first few runs — it is how you tell a grounding failure (clicks landing in the wrong place) from a reasoning failure.

This command opens a real, unsandboxed Chromium on your desktop — the playwright install in step 5 is what put it there. Keep the task on a low-stakes public site, stay signed out, and stay at the keyboard. Fara is trained to stop and ask before entering personal information, making payments, submitting forms, signing in or sending messages; if you see it sail through one of those, stop the run — that is a wiring problem in your prompt path, not a model quirk you should tolerate.

Path B — Magentic-UI, for the sandbox

Everything the model card asks for — the containerised browser, domain allow-lists, watch-mode and a pause control — lives in Magentic-UI, not in fara-cli. This is where a Docker requirement would come from; there is no Docker step in Path A because Path A genuinely does not use one.

Magentic-UI takes the same OpenAI-compatible endpoint you started above, so nothing on this page changes except the client: per the harness README, "instead of fara-cli you can use Magentic-UI". Its setup is documented in its own repository rather than reproduced here, because this recipe has not verified those steps end to end on this GPU — follow microsoft/magentic-ui and give it http://localhost:5000/v1 as the model endpoint.

Results

• Speed: omitted — no published measurement exists for Fara1.5-4B on this card, or on any consumer GPU. The only hardware the vendor names is datacenter class, and borrowing a number from it would be misleading for an 8 GB Ampere card. If you run this, please contribute your numbers so this section can be replaced with a measurement.
• VRAM usage: 3.313 GiB of weights; 4.362 GiB including KV and recurrent state at 32,768 context, derived above from the GGUF's own metadata and llama.cpp's memory formulas. This is a derivation, not an observed peak — compute and vision buffers are additional. See /check/fara1-5-4b/rtx-3060-ti.
• Quality notes: the vendor's published scores (80.8 WebVoyager, 57.3 Online-Mind2Web, 27.4 WebTailBench) are for the BF16 model on datacenter hardware. Q4_K_M is a lossy quantisation of a model whose main job is predicting exact pixel coordinates, and no published evaluation measures how much grounding accuracy that costs. If clicks land near-but-not-on targets, step up to Q6_K (3.425 GiB) or Q8_0 (4.185 GiB) before concluding the model is at fault — both still fit this card with the 32K budget above.
• Language: English only, per the model card's out-of-scope list.

For the full benchmark data, see /check/fara1-5-4b/rtx-3060-ti.

Troubleshooting

ollama run fara does nothing — there is no Ollama entry

There is no Ollama library entry for Fara1.5 in any form: fara1.5, fara, fara1-5 and fara-7b all 404. The GGUF/LM Studio/Ollama line in the harness README refers to Fara-7B, the previous generation, reached with a --fara-7b flag — it is not about Fara1.5, and a lead aggregator has already mis-attributed it. Likewise, the "run them in your choice of tools" list on the quantiser's card is that quantiser's standard boilerplate, printed with the caveat that a newly supported model may need to wait for an update from the developers; it is not a statement that LM Studio or Jan has tested this model. llama.cpp built from source is the path this recipe verifies.

The model answers but ignores the screenshot

Almost always the projector. Confirm --mmproj is on the command line, that the server logged an mmproj load stage, and that Step 4's curl describes the image. A text-only GGUF paired with no projector loads and runs perfectly happily — it simply cannot see, which in a CUA presents as confidently wrong actions rather than an error.

Clicks land in the wrong place

Check the viewport first. The harness defaults to 1440×900 because that is what the model was mostly trained at; a different window size changes both the grounding distribution and the token cost derived above. Then confirm --image-min-tokens 1024 is set, since llama.cpp's own warning ties sub-1024-token images to degraded grounding. Only after both, suspect the quantisation and try a higher tier.

rope.dimension_sections has wrong array length; expected 4, got 3

This is a real open llama.cpp bug (PR #25334), but it belongs to Ollama-exported qwen35 blobs, not to the GGUF this recipe uses. The bartowski artifact's metadata reads qwen35.rope.dimension_sections = [11, 11, 10, 0] — four entries — so it cannot hit this failure. If you see this error, you are loading a different file than the one downloaded in Step 2.

Generation is slower than a 4B model should be

There is an open, community-reported performance regression affecting exactly the SSM kernels this architecture leans on — llama.cpp issue #25162 reports a 24-42% loss on an RTX 2080 Ti (Turing, SM75) after commit 9e58d4d69, and notes the Qwen35 architecture uses linear-attention layers for about 75% of its blocks (which matches the 24-of-32 split derived above). Two caveats before you act on it: it has no maintainer response, and it was measured only on Turing — the RTX 3060 Ti is Ampere (SM86) and nobody has posted numbers for it. Treat it as a lead to check with llama-bench across a couple of builds, not as a known property of this card.

Out of memory at load

Drop to -c 16384 (saves 0.5 GiB of KV, still above the harness's 15,000-token floor), or quantise the KV cache with --cache-type-k q8_0 --cache-type-v q8_0 to roughly halve it. Keep the f16 projector: at 0.626 GiB it is not where the savings are, and the q8_0 projector some quantisers ship trades vision fidelity in a model whose entire job is looking at pixels.