YuE2-3B on RTX 4060 Ti 16GB: the fit transfers, the timings do not

What You'll Build

A complete song — vocals and accompaniment, 48 kHz stereo — generated on one RTX 4060 Ti 16GB from a style prompt and lyrics, together with the editable ABC score the model planned it from, so you can revise the melody or harmony and re-render the same song.

Hardware data: RTX 4060 Ti 16GB (Ada Lovelace, compute capability 8.9, 128-bit GDDR6) · the 16 GB fit is measured, on a card with 1.56× this one's memory bandwidth · See benchmark data

⚠️ This page borrows a measurement from a faster card, and that direction matters — read this before you plan around any number below. The vendor asks for a 24 GB card and publishes nothing for a 16 GB one, so on 2026-09-10 the site operator ran YuE2-3B four times on an RTX 5060 Ti 16GB: two songs at the shipped semantic budget, two filling the model's context to the token, one of those in the mode that also builds the second guidance branch. No run went out of memory. The memory result transfers here in full, because the tensors that decide it are sized from the model's config.json and from a clamp that reads your board's capacity and nothing else. The timings transfer in the wrong direction. Every other 16 GB card in this tier has more memory bandwidth than the card that was measured; this one has less — 288 GB/s against 448 — so where the sibling pages can call the measured seconds an upper bound, this page cannot. Treat them as a floor: a 3.35-minute song took 205.23 s of generation on the faster card, and on this one it will take longer by an amount nobody has measured. If you run this pair, send us your numbers — this is the one card in the tier where that would settle something the arithmetic cannot.

⚠️ The weights are non-commercial. YuE2's code is Apache 2.0, but the checkpoints this recipe downloads are CC BY-NC 4.0. The repository's LICENSE scopes that to "the YuE2 checkpoint weights in model.safetensors, or the corresponding" model files. The licence text names the weights and does not address the audio you generate with them — if you intend to release or monetise output, read the full licence rather than this paragraph.

ℹ️ One runtime, and it is a Python wheel. YuE2-3B runs through the vendor's own yue2 package with CUDA. There is no llama.cpp, Ollama or LM Studio path; a GGUF conversion exists for a different engine and its status changed on the day this page was written — see Troubleshooting.

Requirements

The vendor states its hardware line as "Linux · Python 3.10+ · 24GB NVIDIA GPU with BF16 support." on the model card, and as "NVIDIA GPU with BF16 support and 24 GB VRAM." in the GitHub repository. Both constrain capacity and dtype and nothing else — no generation, no bus width, and certainly no bandwidth floor. The 24GB is a recommendation rather than something the software enforces, which is why somebody opened a request on a low-VRAM inference project the day the model shipped, asking "Any possibility to optimise within 16gb vram?" (Wan2GP issue #2283, 2026-09-10). Nothing needs optimising for capacity. The arithmetic is below; what it cannot tell you is how long you will wait.

Why 16 GB holds, in bytes — and why a longer song cannot change it

The library caps the process, not the card. YuE2Pipeline.__init__ reads the device's total memory and sets budget = min((memory_budget_gib - 2) * 2**30, total - 2 * 2**30), then enforces it with set_per_process_memory_fraction (src/yue2/pipeline.py L161-165). memory_budget_gib defaults to 24 on any card, so on a 16 GB board the total - 2 term binds. total is what CUDA reports for your board, a little under 16 GiB and not necessarily the same number as another board of the same nominal size; the card the runs were made on reported 15.93 GiB, so its ceiling was 13.93 GiB. Read yours out of yue2 doctor in step 2 and subtract 2 rather than assuming 13.93 is yours. Two consequences hold on every board: an out-of-memory error can arrive with memory physically free, and the figure to compare against the ceiling is the process's own peak — not the device peak, which also holds whatever a display or monitoring tool is using and which the clamp never sees.

The KV cache is sized by the budget you ask for, not by how many tokens the model emits. The graph runner preallocates, per layer and once each for keys and values, a tensor shaped (branches, capacity, num_key_value_heads, head_dim) where capacity = max(len(prefix)) + max_tokens (src/yue2/cuda_graph.py L70, L90-L92). From config.json — 28 layers, 8 key/value heads, head dimension 128, dtype bfloat16 — one token on one branch costs 28 × 8 × 128 × 2 bytes × 2 tensors = 114,688 B, i.e. 112 KiB. At the full context that is 2.625 GiB on one branch and 5.25 GiB on two. Not one term in that product is a bandwidth, a clock or a bus width, which is exactly why the slower card in this tier fits the model just as the faster one does.

Two is the hard maximum, and so is the context. The sampler builds either [prefix] or [prefix, negative] — one branch at guidance exactly 1, two otherwise, and never more (src/yue2/sampling.py L92); cot="off" takes the two-branch path because its default guidance is 1.01 (src/yue2/protocol.py L106). A request whose prefix plus budget exceeds 24,576 tokens is rejected outright rather than quietly trimmed — "Prefix + requested generation budget exceeds 24576; no implicit truncation" (src/yue2/sampling.py L62-63) — and the context cannot be lowered: the generation config raises "Require context=24576 and midpoint with positive integer steps" for any other value (src/yue2/protocol.py L54-55).

So the worst case is bounded, and the bound is the case that was measured:

Against a 16 GiB card that leaves 3.987 GiB for activations, the CUDA context and fragmentation — and the measurement lands where that predicts: the full-context two-branch run on the RTX 5060 Ti 16GB peaked at 12.64 GiB in-process, 0.627 GiB above the arithmetic floor, with 1.29 GiB still under that card's ceiling.

A longer song cannot exceed it. Song length past the context limit is realised as more NAR and VAE chunks, not bigger ones: each chunk's width comes from the context and the prefix, size = min((context - prefix_tokens - 3) // 2, CONTEXT) (src/yue2/protocol.py L141-145), and the chunks run one after another. On this card that is worth restating with the emphasis reversed from the sibling pages: a longer song will not cost you memory, and it will cost you disproportionately more time here than on any other card in this tier, because the extra chunks are exactly the work that a narrower memory system slows down.

Installation

1. Install the inference package into a dedicated virtual environment

The venv is not a style preference. The wheel pins all eight of its runtime dependencies with ==, torch==2.10.0 among them, and an == pin does not step aside for a newer version that is already installed — it replaces it. A ComfyUI user reported on 2026-09-10 that installing this wheel into a working portable build rewrote PyTorch 2.11+cu130 down to 2.10 and left the launcher failing to start. There is also no yue2, yue2-infer or yue2_infer package on PyPI, and the vendor's own setup notes name what not to install: "unverified package with a similar name from PyPI." (skills/yue2-music/references/models-and-setup.md).

python3 -m venv .yue2 && source .yue2/bin/activate
python -m pip install -U pip huggingface-hub==0.36.2
hf download m-a-p/YuE2-3B yue2_infer-0.1.5-py3-none-any.whl --local-dir .
python -m pip install ./yue2_infer-0.1.5-py3-none-any.whl

Those middle three lines are the model card's quick start unchanged, and 0.1.5 is the version the card names — the same wheel the runs below were taken on (sha256 8801e2c0…, 66,117 bytes). The repository tree has moved on to 0.1.6; unpacking both and comparing them file by file leaves 12 of 14 modules byte-identical, the differences confined to the version string and to cli.py. Every engine module cited on this page — pipeline.py, cuda_graph.py, sampling.py, protocol.py, storage.py, quantization.py — is identical in both. Install PyTorch's CUDA build first if your environment would otherwise resolve a CPU-only wheel; beyond that there is no wheel selection to get right on this card.

2. Confirm the card, the dependencies, and your own ceiling

yue2 doctor

Read dependencies_ready and the cuda array in the JSON it prints. The array names your board, gives its total memory in GiB — subtract 2 for the ceiling this process will be held to — and reports its compute capability, which for an RTX 4060 Ti is 8.9 (CUDA GPUs). Worth confirming the memory figure rather than assuming it, because this card ships in an 8 GB variant as well, and the 8 GB one is not covered by anything on this page: its ceiling would be 6 GiB against a 12.013 GiB floor. doctor reports environment readiness only, and says as much itself — "Environment readiness is not quality or real-24GB acceptance." (src/yue2/cli.py L78).

3. Let the loader fetch the weights

The first pipeline call downloads what it needs from m-a-p/YuE2-3B and m-a-p/YuE2-Vae. The loader uses an explicit allow-list (src/yue2/storage.py L32-40) rather than a full clone, so the demo audio and images in those repositories are skipped — that is why the Storage row is 7.79 GB and not the 7.83 GB the two repositories hold in total. Weight files are hash-checked against weights_manifest.json as they load. On the reference card that took 30.40–34.81 s across four runs; expect the load step to be slower here too, since it ends in filling 6.763 GiB of device memory over a narrower bus.

Running

Load the pipeline once, then generate:

import json
from pathlib import Path
from huggingface_hub import hf_hub_download
from yue2 import YuE2Pipeline

repo = "m-a-p/YuE2-3B"
demo = json.loads(Path(hf_hub_download(repo, "examples/tonight-awake.json")).read_text(encoding="utf-8"))

pipe = YuE2Pipeline.from_pretrained(repo, device="cuda")
song = pipe(style=demo["style"], lyrics=demo["lyrics"], cot="full", seed=demo["seed"])
song.save("song.flac")
song.save_artifacts("outputs/song")  # ABC, tokens, latents, audio and settings

That is the exact call the two shipped-budget runs below were measured on, with the vendor's own example prompt — which ships inside the model repository and is fetched separately from the weights, because the loader's allow-list does not include it. The two context-filled runs drove the same pipeline one stage at a time (plan → generate_semantic → synthesize → decode) purely so the semantic budget could be set to 24576 - len(plan.prefix) exactly; every other sampling parameter stayed at its shipped default. If you want to see where the time goes on this card, driving the four stages separately is also the cheapest way to find out.

song.flac is the finished stereo song. outputs/song holds the ABC score, the semantic tokens, the acoustic latents and the settings used — edit score.abc and pass it back as abc= to re-render the same song with a revised melody or harmony. That last point is worth more here than on a faster card: re-rendering from an edited score reuses the plan instead of paying for a fresh one.

The same modes are available from the shell, which is the easier path when you want to queue several songs:

yue2 generate --cot full --style "Mandarin funk, nu-disco" --lyrics-file lyrics.txt --output outputs/song

cot="full" plans melody and chords and is the default; cot="melody" plans melody only and is what the vendor recommends for covers; cot="off" generates with no symbolic plan and, as derived above, is the mode that costs a second KV branch. Do not pass --budget here: the board already clamps the default, so 16 changes nothing, and 12 or below lowers the ceiling to 10 GiB and halves the VAE decode tile (src/yue2/pipeline.py L148) — and on the slowest-memory card in the tier, halving the decode tile is not a trade you want to make blind. Plan for one song at a time; the vendor's own framing is "One song at a time.", and the batch subcommand queues requests rather than running them concurrently.

Results

Measured on a 16 GB card — and, uniquely in this tier, a faster one

All four rows were run by the site operator on 2026-09-10 on an RTX 5060 Ti 16GB (Blackwell, compute capability 12.0, driver 591.86, WSL2 Ubuntu 24.04, yue2_infer 0.1.5). Instrumentation was whole-device NVML sampled at 20 Hz — the same instrument the vendor used, and deliberately not torch.cuda.max_memory_allocated(), which excludes both the CUDA context and the allocator's reserved-but-unused blocks and therefore reads low. Each run was a separate process, because PyTorch's caching allocator returns nothing to the driver on its own: after del and gc.collect() the pipeline still held 7.92 GiB, and only torch.cuda.empty_cache() brought it to 1.14 GiB. A display was attached to that card and held 0.76–0.84 GiB throughout, hence both a device peak and a peak-minus-idle figure; the second is the one the clamp applies to. One rig, one operator, one run per configuration.

• VRAM usage — this transfers, and it is the whole reason the page exists. 9.03 GiB in-process for an ordinary song, 12.64 GiB in the worst case the library's own allocation rule permits, no failures, closest approach to a ceiling 1.29 GiB. Every input is either a byte count from the weights, a field in config.json, or your board's capacity minus 2 GiB. A narrower memory bus changes none of them, so an RTX 4060 Ti 16GB is expected to land on the same peaks, give or take the tens of megabytes that activations, the CUDA context and fragmentation account for. Its own peak has never been observed.
• Speed — this does NOT transfer, and unlike its siblings this page cannot even offer a bound. NVIDIA's comparison page lists this card as 16 GB GDDR6 on a 128-bit interface and the measured card as 16 GB GDDR7 on the same 128-bit width (compare specs); board-partner tech specs put the pins at 18 Gbps here against 28 Gbps there (RTX 4060 Ti 16GB, RTX 5060 Ti). Same bus width, older memory generation: 288 GB/s against 448 GB/s, so this card has 0.64× the measured card's bandwidth, or equivalently that card has 1.56× this one's. Every other 16 GB board in the tier sits on a 256-bit bus and is therefore faster than the card that was measured; this is the only one that is slower. So the seconds in the table above are a floor for this card and not a ceiling, and how far above that floor it lands is not derivable — an autoregressive decode is bandwidth-sensitive but not purely bandwidth-bound, and the NAR and VAE stages are a different mix again. Nobody has published a run of YuE2-3B on this card. Send us the numbers if you do; on this card that measurement would answer a question no other page in the tier leaves open.
• The context-filled rows are the point of this page. The vendor reports that "maximum-context testing peaked at" 14.08 GiB on its own hardware — a number which, read as a device peak against a 16 GB card's process ceiling, looks like it rules out a maximum-length song. It does not: the two are measured against different things, and the run that filled the context to the token cleared it with 1.29 GiB spare. The vendor never defines what its maximum-context run was, so this page claims only to have measured the worst case the shipped code can be asked for.
• Quality notes: the vendor reports a 6.7316 SongBench average for YuE2 and 6.9632 for its best-of-8 setting across 192 WildSongBench prompts, using the legacy decoder — its own automatic metrics under its own candidate-selection protocol, not an independent evaluation. Pass vae="m-a-p/YuE2-Vae-legacy" to from_pretrained to reproduce that protocol; the default decoder is m-a-p/YuE2-Vae, which is what the rows above used. Nothing about quality depends on the card.

Every published speed figure belongs to a much larger card

The vendor's reference is an RTX 4090 24GB: 139.48 LM tokens/s, 214.85 s of audio in 71.04 s and an 11.18 GiB NVML peak in cot="full" mode, headlined as "A 3.6-minute song in 71 seconds on an RTX 4090." (model card), with the method note "NVML records the full-run GPU peak." — the same instrument as the table. The only third-party figure that exists is a bare real-time factor: the author of a C++ port measured his own engine at 0.30 against "python 0.31" for the reference implementation on the vendor's example prompt (audio.cpp issue #499, 2026-09-10), naming no card in that comment, no mode and no method. Take the vendor's headline as the shape of what a high-end card does and read this card's position from the bandwidth comparison above rather than from either number. The site operator's runs are the closest published measurement to this card, and they are on a card 1.56× faster.

Nothing is filed at /check/yue2-3b/rtx-4060-ti-16gb yet, so that page is where the first measurement of this pair will land — contribute yours.

Troubleshooting

The unquantized preset requires CUDA BF16 support

This is the pipeline's only hard hardware gate, and it is a BF16 test rather than an architecture test: the constructor calls torch.cuda.is_bf16_supported() and raises that message when it returns false (src/yue2/pipeline.py L158-160). Ada supports BF16 and an RTX 4060 Ti clears it. Nothing else in the shipped package tests compute capability on the default path, and — worth saying explicitly on the tier's slowest card — nothing in it tests bandwidth, clocks or core count either. The model either fits or it does not, and it fits.

An out-of-memory error with memory free on the card

That is the clamp, not the board. set_per_process_memory_fraction holds the process to total - 2 GiB while nvidia-smi still shows free memory: the reference card's two-branch context-filled run peaked with 2.45 GiB physically free and only 1.29 GiB under its own 13.93 GiB ceiling. Two things make it worse and both look like helping: passing --budget 12 (or lower) caps the process at 10 GiB and halves the VAE decode tile, and running a second generation inside the same Python process leaves the previous run's allocations resident, because PyTorch's caching allocator does not return them to the driver. On the reference machine a loaded pipeline held 7.92 GiB, still 7.92 GiB after del and gc.collect(), and 1.14 GiB after torch.cuda.empty_cache(). Call torch.cuda.empty_cache() between songs, or use one process per song. The temptation to keep one process alive across a batch is stronger here, because reloading costs more on a narrow bus — resist it, or call empty_cache().

RuntimeError: USE_FLASH_ATTENTION was not enabled for build. on native Windows

The reference runs were made under WSL2 on Linux wheels; a native Windows install hits this instead. The graph runner's flash-attention eligibility test asks whether torch.ops.aten._flash_attention_forward exists and whether its schema carries seqused_k (src/yue2/cuda_graph.py L75-76) — a question about the operator's registration, not about whether the build compiled its kernel. On a wheel built without USE_FLASH_ATTENTION the operator is registered and then throws when called, which is what a community ComfyUI wrapper documents for official Windows wheels: "The op exists and then crashes" (ComfyUI-YuE2 README). The vendor's own escape hatch is a documented backend rather than a patch: backend="torch-eager" (or --backend torch-eager) is accepted at src/yue2/pipeline.py L128-129 and turns off the CUDA-graph path at L250, so the selection above never runs. Do not pip install flash-attn; the package declares no such dependency and its "FlashAttention" is PyTorch's own kernel. Two caveats, and the second has extra weight on this card: the eager route allocates its per-branch KV cache at len(ids) + max_tokens (src/yue2/sampling.py L76-79), so the memory bound derived above still holds — but giving up the CUDA-graph decode path costs speed on a card that has the least of it in this tier, and nobody has measured how much. Requesting flash explicitly at least fails at construction: attention_backend="flash" raises "Pinned PyTorch variable-length CUDA FlashAttention is unavailable" (src/yue2/cuda_graph.py L82-83).

Memory budget must leave room for a 2GiB reserve

The same clamp on a card too small for it: budget comes out at zero or below once the device has 2 GiB or less. On the 8 GB version of this card the ceiling is 6 GiB against the 12.013 GiB floor derived above, so that variant is out and no setting rescues it. On a 12 GB card the ceiling is 10 GiB — both terms of the min evaluate there, so no --budget value raises it — against the vendor's own 11.18 GiB ordinary-case peak, with the context not reducible and the two candidate levers (--budget 12 --offload-ar) unmeasured by anyone. A card with no documented headroom is not the same claim as a proven failure, and that is why 12 GB stays out until someone runs it.

Raising cfg_scale costs a second KV branch

The model card's tuning table suggests cfg_scale=1.2 to strengthen text guidance, and any guidance other than 1 makes the sampler build a second, unconditional branch whose count is the leading dimension of the KV cache — 2.625 GiB more at full context, as derived above. On 16 GB that is affordable, and the two-branch case is measured rather than assumed: both cot="off" rows run two branches, and the context-filled one is the largest KV allocation the code permits, so raising cfg_scale cannot push that term past the table. It does, however, mean a second prefill and a second stream of decode work, so the cost you will notice here is in seconds rather than in bytes. What the rows cannot price is the second branch alone, because cot="off" also skips the ABC planning stage entirely; change one thing at a time and watch both the peak and the clock.

Experimental FP8 AR requires CUDA compute capability >=8.9

The package has an opt-in FP8 mode for the AR projections, selected with quantization="fp8", and it is the only place in the package that tests compute capability: if torch.cuda.get_device_capability(device) < (8, 9): raise (src/yue2/quantization.py L71-72). An RTX 4060 Ti reports exactly 8.9 (CUDA GPUs), so it passes the gate with nothing to spare. It is tempting to reach for it on the tier's slowest card; do not. The module's own status output calls its quality unvalidated, it is incompatible with the CUDA-graph decode path — so you would be trading a measured decode path for an unvalidated one — and it keeps a second copy of the BF16 weights in host memory. The unquantized model fits, as measured.

The GGUF files on the Hub, and the C++ engine that reads them

A GGUF conversion of this model exists at audio-cpp/Yue2-3B-GGUF, and it is not a llama.cpp or Ollama artifact — its header declares general.architecture = "audiocpp", which is not among llama.cpp's registered architectures. It targets 0xShug0/audio.cpp, a separate ggml-based audio engine, and that port landed on the engine's dev branch while this page was being written: re-checked at head 3caeba87, committed 2026-09-10 at 17:09 UTC, 29 yue2 paths including src/models/yue2/ and docs/models/yue2.md. Its author reports a peak below 9 GB and a real-time factor of 0.23 to 0.28 through that engine's interface on a 32 GB card, noting that "The numbers are a bit noisy because of background workloads" (vendor issue #163). This is the entry most worth reading carefully from the tier's slowest card, because a quantised C++ path is exactly the kind of thing that sounds like the answer to a speed problem — and none of those figures is evidence about this card. It is a different engine on a Q8 quantisation, self-reported by the port's author, on a card with twice this one's memory, unverified by us; and it is the deliberately smaller of his configurations, since in the same thread he says a more aggressive cache would take the real-time factor below 0.2 at higher VRAM. By his own announcement the model sits in a branch for community testing that "won’t be included in the prebuilt binaries before merging" (discussion 3). Build dev and expect to be an early tester if you want to try it. This page documents the vendor's Python runtime, which is what every figure above was measured on.

Community reports so far

YuE2-3B was published on 2026-09-09. Re-checked on 2026-09-10 at 21:00 UTC: m-a-p/YuE2-3B carries three discussion threads, all opened the previous day — a question about getting reliably instrumental output, a ComfyUI node-pack announcement, and the audio.cpp port above — and m-a-p/YuE2-Vae carries none. The same query against the org's YuE v1 model returns ten threads, so the near-silence is the model's age rather than a broken check. A GitHub-wide issue search for YuE2 returns 22 items: the vendor's own release pull requests, three requests to port the model to other engines, and two bugs in a community ComfyUI wrapper. Reading the title and body of all 22, all three discussion threads in full, and the comment threads on the port requests: no 60-class card appears, and no 16 GB board of any kind — so if you are wondering what this card does with the model, the honest answer is that this page's arithmetic says it fits and nobody on the public record has timed it. Every hardware-shaped issue in the vendor's tracker that a search for "YuE" surfaces — a ROCm report, Apple Silicon requests, GGUF requests — predates YuE2 and was filed against YuE v1, a different model with a different runtime; do not transfer their answers here. Nothing in the current package mentions ROCm, HIP or Metal, and the vendor documents no AMD path.

If you hit something, please report it via the submission form.