How much VRAM does Fara1.5-4B need?

About 16 GB — the minimum this recipe targets.

How hard is this setup?

Advanced — follow the steps above.

Fara1.5-4B on RTX 4080 Super: BF16 Browser Computer-Use Agent under llama-server

What You'll Build

A local browser computer-use agent (CUA): Microsoft's Fara1.5-4B served by llama-server on RTX 4080 Super, with the vision projector loaded so the model actually sees browser screenshots, driven by Microsoft's own fara-cli harness against a plain local Playwright Chromium.

The 16 GB on this card buys something specific, and it is not more speed at the same settings — it is a different lead configuration. Where the smaller cards in this catalogue spend their whole budget deciding how far to quantise a model that predicts pixel coordinates, this one runs the unquantised BF16 weights and still has a 65,536-token context and several gigabytes to spare. The trade it cannot make is having that and the model's native 262,144-token window; the table below shows why.

Hardware data: RTX 4080 Super (16GB VRAM) · BF16 weights + f16 mmproj = 8.472 GiB, 10.668 GiB accountable at 65,536 context (derived below) · See benchmark data

⚠️ This model clicks, types and submits in a real browser, and the path documented here gives it no isolation of any kind. That is the operating requirement, not boilerplate, and it is the thing to settle before anything else on this page.

The model card is blunt about the consequence: "Don't run this model with unrestricted browser access on a machine that has anything sensitive on it." That sentence closes the card's Recommended Deployment: MagenticLite section, which lists sandboxing, allow-lists, watch-mode and a pause control as MagenticLite features and then says: "If you integrate Fara1.5-4B directly, you're responsible for these controls."

fara-cli is a direct integration. Its install step is playwright install, and it drives a plain local Playwright Chromium — no container, no allow-list, no pause control. The harness README offers the containerised option as an alternative to it, not as part of it: "instead of fara-cli you can use Magentic-UI", described there as "a sandboxed browser environment with auditable action logging and user prompts at critical points". For the CLI path the vendor's position is a recommendation, not something shipped: "We recommend running it in a sandboxed environment, monitoring its execution, and avoiding sensitive data or high-risk domains."

So on the fara-cli path you are the boundary. Practically: a throwaway OS user or a VM, Playwright's own fresh browser profile (never your daily one), signed out of everything, low-stakes public sites, and eyes on the run rather than leaving it unattended. The command in Running passes --headful for exactly that last reason. If you want the container, the allow-lists, watch-mode and the pause control, use Magentic-UI as the client instead.

ℹ️ Vision is the product, not a bonus. Per the model card, "The model is vision-only at perception time: it sees the browser through screenshots, not the DOM or accessibility tree." A Fara install that only serves text is a broken install — the --mmproj step below is mandatory, and Step 4 exists to prove that an image really reached the model before you let it drive anything.

⚠️ Runtime status — read before you commit an evening to this. llama.cpp support for this architecture is verifiable statically, and this recipe shows that evidence rather than asserting it. What does not exist anywhere public is a report of Fara1.5 decoding an image under llama-server or llama-mtmd-cli. The closest datapoint is a community user serving Fara1.5-27B under llama-server on build b10107 and driving it from an agent harness successfully — but that thread is about tool-call parsing, not about the projector. Treat your first run as the test, and see Results for why there is no speed figure here either.

Requirements

Component	Minimum	This recipe
GPU	16GB VRAM, CUDA compute 8.9 (Ada Lovelace)	RTX 4080 Super (16GB) — not measured; the budget below is derived (contribute a run)
RAM	16GB (weights are mmap'd on their way to VRAM; Playwright's Chromium sits on top)	—
Storage	9.10 GB (BF16 + f16 mmproj)	9,096,817,152 B = 8.472 GiB, plus the harness and Chromium
Software	llama.cpp b10087+ (CUDA), Python 3.10+, Playwright	—

Microsoft ships safetensors only, and all of it in BF16: HuggingFace's tensor census for the repo reports one dtype and one dtype only, {"BF16": 4539265536}, and the two shards total 8.455 GiB per the HF tree API (5,960,243,904 + 3,118,376,552 B). Its Quickstart serves those with vLLM and names datacenter hardware for the job: "A GPU with enough memory for a 4B model in bf16 (A6000, A100, H100, and B200 have been tested)", and the card's highlights claim only that the model "runs on a single A100/H100/B200 with room for the screenshot history". What this recipe shows is that the precision in that sentence, at least, is available on a 16GB consumer card.

What 16 GB actually buys

There is no benchmark for this pair yet — /check reports unknown with zero benchmark rows — so every number below is arithmetic from the artifacts' own metadata and llama.cpp's source, not a measurement. Reproduce it rather than trusting it.

Weights. Byte counts from the bartowski GGUF tree:

Component	Bytes	GiB
`Fara1.5-4B-bf16.gguf`	8,424,393,568	7.846
`mmproj-Fara1.5-4B-f16.gguf`	672,423,584	0.626
Weights total	9,096,817,152	8.472

KV cache — and this is where Fara1.5 is unusual. Its qwen35 architecture is a hybrid: llm_arch_is_hybrid() in src/llama-arch.cpp returns true for LLM_ARCH_QWEN35, and src/models/qwen35.cpp marks layers recurrent with hparams.is_recr_impl[i] = (i < hparams.n_layer()) && ((i + 1) % full_attn_interval != 0). The BF16 file's own header carries qwen35.block_count = 32 and qwen35.full_attention_interval = 4, and the source config.json spells the split out layer by layer: its layer_types array holds 24 linear_attention entries and only 8 full_attention. Only those 8 carry a cache that grows with context.

With qwen35.attention.head_count_kv = 4 and attention.key_length = attention.value_length = 256:

K+V per token per attention layer = 2 x 4 x 256 x 2 B (f16) = 4,096 B
x 8 full-attention layers                                   = 32 KiB / token

A non-hybrid 32-layer model with the same head geometry would cost four times that. Context on this model is cheap — doubling -c costs exactly 1.000 GiB per doubling from 32,768 — and that is the fact the whole budget turns on.

Recurrent state — fixed, and tiny. The other 24 layers hold a context-independent state, sized by n_embd_r() / n_embd_s() in src/llama-hparams.cpp from the artifact's own GGUF keys (ssm.conv_kernel 4, ssm.inner_size 4096, ssm.group_count 16, ssm.state_size 128). qwen35 sets no n_embd_head_kda, so both fall through to the Mamba forms:

conv state : (4-1) x (4096 + 2 x 16 x 128) =  24,576 elems
ssm  state :         128 x 4096            = 524,288 elems
per layer  : 548,864 x 4 B (F32)           = 2.094 MiB
x 24 layers                                = 50.25 MiB per sequence
x 4 sequences                              = 201 MiB = 0.196 GiB   <- does not grow with context

That last multiplication is easy to miss and it is not optional. src/llama-model.cpp sizes the recurrent state with /* recurrent_rs_size */ std::max((uint32_t) 1, cparams.n_seq_max), and tools/server/server.cpp resolves the server's auto default with params.n_parallel = 4, so a plain llama-server reserves four sequences' worth whether or not you use them. The KV cache is not multiplied the same way: that same default also sets params.kv_unified = true, so the 32 KiB/token derived above is the whole KV cost regardless of slot count.

The whole trade, in one table. Accountable allocations (text weights + f16 projector + KV + recurrent state) in GiB, on a card with 16 GB, each cell summed from exact byte counts and rounded once:

Context	Q4_K_M	Q5_K_M	Q6_K	Q8_0	BF16
32,768	4.509	4.910	5.247	6.008	9.668
65,536	5.509	5.910	6.247	7.008	10.668
131,072	7.509	7.910	8.247	9.008	12.668
262,144 (native)	11.509	11.910	12.247	13.008	16.668

Read that table as three facts rather than sixteen numbers.

BF16 stops being a compromise here, and that is the headline. The 12GB recipe for this model leads Q8_0 and explains on its own page why it declines BF16: there, BF16 forces the context back down to 32,768 and still accounts for 9.668 GiB, leaving barely two gigabytes for compute buffers, the vision graph and a co-resident Chromium. On 16 GB the same weights at twice that context account for 10.668 GiB and leave roughly 5.3 GiB. So this is the first card in this catalogue where you get the vendor's own precision without paying for it somewhere else — and the precision is exactly where the risk on this model sits, because Fara's output is pixel coordinates, its published scores are for BF16 weights, and no public evaluation measures what any quantisation tier costs a grounding model. At Q8_0 that risk is small; at BF16 it is not a question you have to hold open at all.
The native 262,144-token window becomes reachable — but only if you quantise. At Q8_0 it lands at 13.008 GiB, with about 3.0 GiB left. On a 12GB card no tier reaches it: even Q4_K_M is 11.509 GiB there, before a single compute buffer. This is the second thing 16 GB buys, and it is real.
You cannot have both. BF16 at 262,144 is 16.668 GiB, which is over the card outright. So the decision on this card is not which quant — it is precision or window, and this recipe picks precision, for the reason in the first bullet. If your trajectories genuinely run long, BF16 at 131,072 is the middle option at 12.668 GiB and still fits.

Nothing in that table is free space you can plan against to the last byte: the CUDA context, compute buffers and the vision encoder's working set are allocated at load time and this recipe does not derive them. Read the CUDA0 compute buffer size lines llama-server prints on startup for the real figures on your build, and report what you see.

Two honest costs of the BF16 lead. First, the file is 1.875x the bytes of the Q8_0 build (8,424,393,568 / 4,493,954,144), and single-token decode on a model this small is bound by how many bytes leave memory per token, so the BF16 path is the slower one; nobody has measured Fara1.5 on any consumer GPU, so this recipe publishes a direction and not a ratio. Second, it is 9.10 GB to download instead of 5.17 GB — both paths pull the same 672,423,584 B projector, so the extra is 3.93 GB of weights, not 4.61. If either matters more to you than the open question about quantised grounding, Fara1.5-4B-Q8_0.gguf is a one-word substitution in Step 2 and Step 3 and leaves every other line on this page unchanged.

Why the f16 projector next to BF16 text weights. bartowski publishes both a BF16 and an f16 build of the vision tower, and they differ by exactly 3 MiB (675,569,312 vs 672,423,584 B), so this is not a budget decision. At the same two bytes per value, IEEE binary16 spends three more bits on the mantissa than bfloat16 does and three fewer on the exponent — a finer grid over a narrower range. This recipe takes the f16 build, which is also what every other recipe for this model uses. If you would rather keep the source dtype byte-for-byte, substitute mmproj-Fara1.5-4B-bf16.gguf and add 3 MiB to every figure above.

BF16 is a first-class type on this backend, not a fallback

Worth establishing before you download nine gigabytes of it, because "GGUF" and "supported" are not the same claim. Three checkable places in llama.cpp's CUDA backend:

The hardware predicate. bf16_mma_hardware_available() in ggml/src/ggml-cuda/common.cuh returns true for GGML_CUDA_CC_IS_NVIDIA(cc) && cc >= GGML_CUDA_CC_AMPERE, i.e. compute capability 800 and up. The same file defines GGML_CUDA_CC_ADA_LOVELACE as 890. These cards clear the runtime's own BF16 tensor-core check.
The decode kernel. ggml_cuda_should_use_mmvf() in ggml/src/ggml-cuda/mmvf.cu has an explicit case GGML_TYPE_BF16 with its own NVIDIA branch, and the kernel body reads weights through nv_bfloat162 pairs. Single-token generation from BF16 weights runs a purpose-built matrix-vector kernel; it is not converted to F16 first.
The batched path, and the one line where architecture shows up. Prompt processing dispatches ggml_cuda_mul_mat_cublas_impl<GGML_TYPE_BF16> in ggml/src/ggml-cuda/ggml-cuda.cu, whose traits map BF16 to CUDA_R_16BF with CUBLAS_COMPUTE_32F accumulation. A few lines further down, prefer_f32_output = !GGML_CUDA_CC_IS_RDNA3(cc) && !GGML_CUDA_CC_IS_CDNA(cc) is true on any NVIDIA device, which makes the GEMM write its result straight out as F32 instead of into a BF16 temporary that then has to be converted. That is a choice of write target inside the BF16 path — accumulation is F32 either way — and not a fallback out of it.

The artifact matches. Reading the header of the exact file downloaded in Step 2 gives general.architecture = qwen35, general.file_type = 32 (LLAMA_FTYPE_MOSTLY_BF16), and a tensor census of 426 tensors — 249 of type 30 (GGML_TYPE_BF16) and 177 of type 0 (GGML_TYPE_F32, the norms and biases) — spanning blk.0 through blk.31 with no gaps.

How the context is actually spent

A CUA sends a screenshot every step, so the per-image cost decides whether the budget above is realistic. It is not the 768 in the projector metadata: that value derives from the vision tower's position-embedding grid, and llama.cpp treats this projector as dynamic-resolution — --image-min-tokens and --image-max-tokens are documented as "only used by vision models with dynamic resolution".

The harness fixes the browser viewport at 1440x900 (viewport_width: int = 1440, viewport_height: int = 900 in fara15_agent.py), matching the card's note that "The screen resolution Fara is most commonly trained with is" 1440x900. It pre-resizes with Qwen's smart_resize at PATCH_SIZE = 16 and MERGE_SIZE = 2, so the factor is 32 and 900 becomes 896. llama.cpp then counts patches the same way — x_patch = img->nx() / (params.patch_size * 2), y_patch = img->ny() / (params.patch_size * 2) in tools/mtmd/clip.cpp:

(1440 / 32) x (896 / 32) = 45 x 28 = 1,260 tokens per screenshot
x max_n_images = 3                 = 3,780 tokens resident

The harness's own conservative default is image_token_estimate: int = 1500, which is a useful sanity check on the arithmetic, and the model card agrees on the retention: "We only keep the most recent 3 screenshots" in the chat history.

So images are capped. What grows is the text trajectory: every round appends the model's reasoning, its tool call and an observation (max_observation_chars: int = 1000), and fara-cli defaults to --max_rounds 100. That is why 65,536 is not a spec-sheet number — it is 4.3x the floor the harness README asks for explicitly, "Please ensure that context length is set to at least 15000 tokens and temperature to 0 for best results.", with enough slack that a hundred-round run is not the thing that truncates. Microsoft's own framing of the model's 262K window is "Long enough for multi-screenshot trajectories with full action history."

Installation

1. Build llama.cpp with CUDA

Ada Lovelace is sm_89. Build from a release at or after b10087 — the release this GGUF was quantized on, published 2026-07-22.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=89 -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

Both halves of the model are first-class in mainline — worth confirming before you download 9 GB:

grep -n 'LLM_ARCH_QWEN35,' src/llama-arch.cpp        # -> { LLM_ARCH_QWEN35, "qwen35" }
grep -n 'qwen3vl_merger' tools/mtmd/clip-impl.h      # -> PROJECTOR_TYPE_QWEN3VL

2. Download the weights and the projector

Both files are required. The projector is a separate artifact — downloading only the text GGUF gives you a model that cannot see.

pip install -U huggingface_hub

hf download bartowski/Fara1.5-4B-GGUF \
  Fara1.5-4B-bf16.gguf mmproj-Fara1.5-4B-f16.gguf \
  --local-dir ~/models/fara1.5-4b

Microsoft publishes no first-party GGUF for Fara1.5 — the microsoft org ships exactly Fara1.5-4B, Fara1.5-9B and Fara1.5-27B as safetensors, plus an ONNX repo for the previous-generation Fara-7B, which is a different model and should never be mixed in. Four independent community quantisers have converted the 4B including its vision tower — bartowski, prithivMLmods, DevQuasar and runanywhere each publish a matching mmproj, and three of them also publish a full-precision text GGUF. Four independent conversions succeeding is the strongest available evidence that the vision path converts cleanly; it is still not a runtime test. bartowski's card labels the file this recipe uses "Full BF16 weights."

3. Serve it

./build/bin/llama-server \
  -m ~/models/fara1.5-4b/Fara1.5-4B-bf16.gguf \
  --mmproj ~/models/fara1.5-4b/mmproj-Fara1.5-4B-f16.gguf \
  -ngl 99 -c 65536 --host 127.0.0.1 --port 5000 \
  --image-min-tokens 1024 \
  --reasoning-format none \
  --temp 0

Three of those flags are load-bearing and none of them is a default:

--reasoning-format none — mandatory. Without it the agent loop silently receives nothing; see Troubleshooting for the mechanism.
--image-min-tokens 1024 — this projector ships no minimum-pixel key, so llama.cpp falls back to the floor set by hparams.set_limit_image_tokens(8, 4096) in the PROJECTOR_TYPE_QWEN3VL branch of tools/mtmd/clip.cpp and warns, a few lines later in the same branch: "Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks". Fara is a grounding model, so that warning is aimed precisely at this case. At the harness's viewport the image is already 1,260 tokens and the floor never binds; it protects you if you shrink the viewport.
-c 65536 — see the budget table. Do not leave it at the model's native 262,144: at BF16 that is 16.668 GiB and will not load.

If you would rather not manage files, -hf bartowski/Fara1.5-4B-GGUF:BF16 fetches the model, and per the server docs "mmproj is also downloaded automatically if available". One detail before you rely on it: that tag's manifest pairs the text weights with the bf16 projector — its layer list is an 8,424,393,568 B model plus a 675,569,312 B projector, not the 672,423,584 B f16 build Step 2 downloads. So the shortcut lands exactly 3 MiB above every figure in the table above. That is the same substitution the projector note already offers and it changes nothing that matters; the two paths simply are not byte-identical.

4. Prove the vision path works before going further

Check that the server advertises multimodal support, then send a real image. The server docs make the first part a client-side requirement rather than an optional nicety: "Clients should check /models or /v1/models for the multimodal capability before a multimodal request." Run GET /props alongside it — it reports which modalities loaded rather than merely that some did, which is the more useful signal when a projector silently failed to load. For the image itself, image_url.url "can be a remote URL, base64 (raw or URI-encoded via data:image/...;base64) or path to local file".

curl -s http://127.0.0.1:5000/v1/models | grep -o multimodal

# supplementary — names the modality that actually loaded
curl -s http://127.0.0.1:5000/props | python3 -c "import sys,json; print(json.load(sys.stdin)['modalities'])"
# expect: {'vision': True}

curl -s http://127.0.0.1:5000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "fara",
  "temperature": 0,
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "Describe this image in one sentence."},
    {"type": "image_url", "image_url": {"url": "https://raw.githubusercontent.com/ggml-org/llama.cpp/master/media/llama0-logo.png"}}
  ]}]
}' | python3 -m json.tool

If the reply describes the picture, the projector is wired up. If /v1/models never prints multimodal, if /props reports vision: False, or if the model answers as though no image were present, stop here — the agent loop below will produce confidently wrong actions rather than fail loudly.

5. Install the harness

git clone https://github.com/microsoft/fara.git
cd fara
python3 -m venv .venv && source .venv/bin/activate
pip install -e .
playwright install

Skip the README's [vllm] extra — that path serves BF16 safetensors on datacenter cards, which is not how this recipe gets to BF16.

Running

Two clients share the same llama-server endpoint. Path A (fara-cli) is the smoke test, with no isolation. Path B (Magentic-UI) is what the vendor recommends for anything real. They are not equivalent in risk; pick deliberately.

Path A — `fara-cli`, for a first smoke test

fara-cli speaks to any OpenAI-compatible endpoint. Its built-in default already points at http://localhost:5000/v1, but name the model explicitly, since DEFAULT_ENDPOINT_CONFIG in run_fara.py names a different size ("model": "Fara1.5-9B"):

source .venv/bin/activate
fara-cli \
  --base_url http://127.0.0.1:5000/v1 \
  --api_key sk-no-key-required \
  --model fara \
  --task "find the current UTC time on time.is and tell me" \
  --output_folder ./runs/first \
  --save_screenshots \
  --headful \
  --max_rounds 15

--headful is in that command on purpose. The harness runs the browser headless by default — run_fara.py declares the flag at line 230 with the help text "Run the browser in headful mode (show GUI, default is headless)" and both launch sites, lines 321 and 334, read headless=not args.headful. The safety callout above asks you to keep eyes on the run, and headless is the one mode in which you cannot; drop the flag once you trust the loop and are reading the trajectory files instead. Each step writes a screenshot and a data_point.json into --output_folder, which is how you tell a grounding failure (clicks landing in the wrong place) from a reasoning failure.

Keep the task on a low-stakes public site and stay signed out. Fara is trained to stop and ask before entering personal information, making payments, submitting forms, signing in or sending messages; if you see it sail through one of those, stop the run — that is a wiring problem in your prompt path, not a quirk to tolerate.

Path B — Magentic-UI, for the container

Everything the model card assumes — the containerised browser, domain allow-lists, watch-mode and a pause control — lives in Magentic-UI, not in fara-cli. That is also where a Docker requirement would come from; there is no Docker step in Path A because Path A genuinely does not use one.

Magentic-UI takes the same OpenAI-compatible endpoint you started above, so nothing on this page changes except the client. Its setup is documented in its own repository rather than reproduced here, because this recipe has not verified those steps end to end on this GPU — give it http://127.0.0.1:5000/v1 as the model endpoint. Whichever client you use, match the trained resolution: the card is explicit — "Match this in your sandbox for the most reliable grounding."

Results

Speed: omitted — no published measurement of Fara1.5-4B exists for this card or for any consumer GPU, and the only hardware Microsoft names is datacenter class. Borrowing a figure from an A100 for a 16GB Ada card would be misleading, and the vendor publishes no throughput numbers to borrow in the first place. Any 16GB Ada card lands on the same memory budget as this one, because the derivation above depends only on capacity and on sm_89 — but throughput is exactly what those cards do not share, and with no measurement on any of them this recipe does not rank them. If you run this, please contribute your numbers so this line can be replaced with a measurement.
VRAM usage: 8.472 GiB of weights; 10.668 GiB accountable including KV and the four-sequence recurrent state at 65,536 context, derived above from the GGUF's own metadata and llama.cpp's memory formulas. This is a derivation, not an observed peak — compute and vision buffers sit on top. Track the empirical picture at /check/fara1-5-4b/rtx-4080-super.
Quality notes: the vendor's published scores (80.8 WebVoyager, 57.3 Online-Mind2Web, 27.4 WebTailBench) are for the BF16 model on datacenter hardware. The hardware does not transfer; the precision does — that is the point of leading BF16 on this card. No public evaluation measures what quantisation costs a model whose job is predicting exact pixel coordinates, and on 16 GB you do not have to find out.
Language: English only — the card puts "Languages other than English (training data is English-only)" out of scope.

For the full benchmark data, see /check/fara1-5-4b/rtx-4080-super.

Troubleshooting

The agent does nothing — every step logs an empty response or `list index out of range`

This is the failure everyone hits, and it is not a bug in your setup. Fara opens a <think> block and never closes it — you can read that straight out of the chat template embedded in the GGUF you downloaded, whose generation prompt emits '<think>\n' with no closing tag unless enable_thinking is explicitly false. Under llama-server's default reasoning extraction the parser therefore treats the whole output — reasoning and the <tool_call> block together — as reasoning, returns it in reasoning_content, and leaves message.content empty. The harness parses message.content for <tool_call>, finds nothing, and the loop dies on round one.

Launch with --reasoning-format none, documented as "none: leaves thoughts unparsed in message.content". The raw output then begins with a stray unclosed <think> tag, which is cosmetic.

Reported for Fara1.5-27B by a community user on build b10107 in the model's HF discussions and acknowledged there by Microsoft — swhitehead, whose HF account carries the org-member flag on that repo, replied "Would you mind posting this as an issue on the Fara1.5 github repo so we can mention this in the hosting instructions?", which became microsoft/fara#82 — still open, titled "Hosting: llama.cpp needs --reasoning-format none". The same reporter adds: "If you have a router/proxy in front doing its own reasoning extraction, disable it there too."

The model answers but ignores the screenshot

Almost always the projector. Confirm --mmproj is on the command line, that the server logged an mmproj load stage, that /v1/models advertises the multimodal capability, and that Step 4's /props call reports vision: True. A text-only GGUF with no projector loads and runs perfectly happily — it simply cannot see, which in a CUA presents as confidently wrong actions rather than an error.

Clicks land in the wrong place

Check the viewport first: the harness defaults to 1440x900 because that is what the model was mostly trained at, and a different window size changes both the grounding distribution and the token cost derived above. Then confirm --image-min-tokens 1024 is set, since llama.cpp's own warning ties sub-1024-token images to degraded grounding. Only after both should you suspect the weights — and on the lead configuration here you are running the unquantised BF16 conversion of the weights the vendor evaluated, so the quant ladder is not available as an explanation at all. That is a diagnostic advantage of this card worth keeping: it removes one variable from every grounding bug you file.

Out of memory at load

In order of what to try. Drop -c from 65536 to 32768 — that frees exactly 1.000 GiB and is still more than twice the harness's 15,000-token floor. If that is not enough, the next step is the quant ladder, and from BF16 the first rung is large: Fara1.5-4B-Q8_0.gguf frees 3.661 GiB, Fara1.5-4B-Q6_K.gguf 4.421 GiB, Fara1.5-4B-Q5_K_M.gguf 4.758 GiB. Keep the f16 projector: at 0.626 GiB it is not where the savings are, and the q8_0 projector that one quantiser ships trades vision fidelity in a model whose entire job is looking at pixels. If you need the projector out of VRAM entirely, --no-mmproj-offload keeps it on the CPU at the cost of slower image encoding — once per agent turn.

Don't reach for speculative decoding to fill the headroom

A card with several spare gigabytes invites it, and the model family does carry a multi-token-prediction layer — the source config.json declares mtp_num_hidden_layers: 1 inside text_config. This GGUF does not. Its header has no nextn_predict_layers key (llama.cpp's name for it, per LLM_KV_NEXTN_PREDICT_LAYERS in src/llama-arch.cpp), and none of its 426 tensors is a blk.*.nextn.* tensor: the census stops cleanly at blk.31. The MTP layer was not converted, so --spec-type has nothing to attach to on this file. Nothing on this page enables it; if you go looking, that is why it does nothing.

`rope.dimension_sections has wrong array length; expected 4, got 3`

This is a real open llama.cpp bug (PR #25334, titled "fix: load qwen35 Ollama GGUF exports"), but it belongs to Ollama-exported qwen35 blobs, not to the GGUF this recipe uses. The BF16 artifact's own header reads qwen35.rope.dimension_sections = [11, 11, 10, 0] — four entries — so it cannot hit this failure. If you see this error, you are loading a different file than the one downloaded in Step 2.

`ollama run fara` does nothing — there is no Ollama entry

There is no Ollama library entry for Fara1.5 in any form: fara1.5, fara, fara1-5 and fara1.5-4b all 404, and so does the registry manifest API. The GGUF/LM Studio/Ollama line in the harness README refers to Fara-7B, the previous generation, reached with a --fara-7b flag — it is not about Fara1.5. Likewise, the "run them in your choice of tools" list on the quantiser's card is that quantiser's standard boilerplate. llama.cpp built from source is the path this recipe documents.

Generation is slower than a 4B model should be

Two things to separate before you go bug-hunting. The first is expected: BF16 moves 1.875x the bytes per token that Q8_0 does, and that is a cost you chose in exchange for skipping the quantisation question — A/B against Fara1.5-4B-Q8_0.gguf with llama-bench before treating it as a defect. The second is a lead rather than a known property: llama.cpp issue #25162 is an open, community-reported regression against exactly the SSM kernels this architecture leans on, titled "Performance regression on Turing GPUs after 9e58d4d69 — Qwen35 SSM kernels affected". Read the scope carefully: the title says Turing, the reporter measured on an RTX 2080 Ti, the thread has two comments and both are from community accounts with no maintainer response, and nobody has posted numbers for an Ada card. It is a thing to check with llama-bench across a couple of builds, not a property of this one.