YuE2-3B on RTX 5060 Ti: 16GB measured, worst case included

What You'll Build

A complete song — vocals and accompaniment, 48 kHz stereo — generated on one 16 GB RTX 5060 Ti from a style prompt and lyrics, together with the editable ABC score the model planned it from, so you can revise the melody or harmony and re-render the same song.

Hardware data: RTX 5060 Ti (16GB VRAM, Blackwell sm_120) · measured 9.03 GiB in-process for an ordinary song and 12.64 GiB at full context · 201.04 s of audio in 205.23 s · See benchmark data

ℹ️ This page is a measurement, not a derivation — here is exactly what was measured and how far to trust it. Four runs on one RTX 5060 Ti 16GB on 2026-09-10, by the site operator: two with the shipped semantic budget (cot="full" and cot="off") and two more that fill the model's context to the token, one of them in the mode that also builds the second guidance branch. No run out of memory. Instrumentation: whole-device NVML sampled at 20 Hz — the same instrument the vendor used, and deliberately not torch.cuda.max_memory_allocated(), which excludes both the CUDA context and the allocator's reserved-but-unused blocks and therefore reads low. Each run was a separate process, because PyTorch's caching allocator returns nothing to the driver on its own: after del and gc.collect() the pipeline still held 7.92 GiB, and only torch.cuda.empty_cache() brought it to 1.14 GiB — so a second run in the same process reads the first run's memory as its idle baseline and reports a peak that is several GiB too low. Environment: WSL2 Ubuntu 24.04 on Windows 11, yue2_infer 0.1.5, torch 2.10.0+cu128, CUDA 12.8, driver 591.86, and the card was not headless — a display was attached and held about 0.8 GiB throughout, which is why both a device peak and a peak-minus-idle figure are reported below. One rig, one operator, one evening, one run per configuration — unreplicated by construction, and not comparable to the vendor's figures, which average 32 warm requests per mode. If you run this pair, send us your numbers; a second card is the most valuable thing this page is missing.

⚠️ The weights are non-commercial. YuE2's code is Apache 2.0, but the checkpoints this recipe downloads are CC BY-NC 4.0. The repository's LICENSE scopes that to "the YuE2 checkpoint weights in model.safetensors, or the corresponding" model files. The licence text names the weights and does not address the audio you generate with them — if you intend to release or monetise output, read the full licence rather than this paragraph.

ℹ️ One runtime, and it is a Python wheel. YuE2-3B runs through the vendor's own yue2 package with CUDA. There is no llama.cpp, Ollama or LM Studio path; a GGUF conversion exists for a different engine and its status changed on the day this page was written — see Troubleshooting.

Requirements

The vendor states its hardware line as "Linux · Python 3.10+ · 24GB NVIDIA GPU with BF16 support." on the model card, and as "NVIDIA GPU with BF16 support and 24 GB VRAM." in the GitHub repository. That 24GB is a recommendation, not a floor the software enforces — and it overshoots for a reason worth understanding before you trust a 16 GB card with a long song.

Why 16 GB holds, in bytes — and why a longer song cannot change it

The library caps the process, not the card. YuE2Pipeline.__init__ reads the device's total memory and sets budget = min((memory_budget_gib - 2) * 2**30, total - 2 * 2**30), then enforces it with set_per_process_memory_fraction (src/yue2/pipeline.py L161-165). memory_budget_gib defaults to 24 whatever card you have, so on this 15.93 GiB card the total - 2 term binds and the ceiling is 13.93 GiB. Two consequences: an out-of-memory error here can arrive with memory physically free on the board, and the figure to compare against that ceiling is the process's own peak — not the device peak, which also contains whatever a display or a monitoring tool is holding and which the clamp never sees.

The KV cache is sized by the budget you ask for, not by how many tokens the model emits. The graph runner preallocates, per layer and once each for keys and values, a tensor shaped (branches, capacity, num_key_value_heads, head_dim) where capacity = max(len(prefix)) + max_tokens (src/yue2/cuda_graph.py L70, L90-L92). From config.json — 28 layers, 8 key/value heads, head dimension 128, bfloat16 — one token on one branch costs 28 × 8 × 128 × 2 bytes × 2 tensors = 114,688 B, i.e. 112 KiB. At the full context that is 2.625 GiB on one branch and 5.25 GiB on two.

Two is the hard maximum, and so is the context. The sampler builds either [prefix] or [prefix, negative] — one branch at guidance exactly 1, two otherwise, and never more (src/yue2/sampling.py L92); cot="off" takes the two-branch path because its default guidance is 1.01 (src/yue2/protocol.py L106). A request whose prefix plus budget exceeds 24,576 tokens is rejected outright rather than quietly trimmed — "Prefix + requested generation budget exceeds 24576; no implicit truncation" (src/yue2/sampling.py L62-63) — and the context itself cannot be changed: the generation config raises "Require context=24576 and midpoint with positive integer steps" for any other value (src/yue2/protocol.py L54-55).

So the worst case is bounded, and it is the case that was measured:

Against a 16 GiB card that leaves 3.987 GiB for activations, the CUDA context and fragmentation — and the measurement lands where that predicts: the full-context two-branch run peaked at 12.64 GiB in-process, 0.627 GiB above the arithmetic floor, with 1.29 GiB still under the 13.93 GiB ceiling.

A longer song cannot exceed it. Song length past the context limit is realised as more NAR and VAE chunks, not bigger ones: each chunk's width is computed from the context and the prefix, size = min((context - prefix_tokens - 3) // 2, CONTEXT) (src/yue2/protocol.py L141-145), and the chunks are processed one after another. More audio therefore costs more time, not more memory, and the two tensors that dominate the peak are already at their maximum in the table above.

Installation

1. Install the inference package into a dedicated virtual environment

The venv is not a style preference. The wheel pins all eight of its runtime dependencies with ==, torch==2.10.0 among them, and an == pin does not step aside for a newer version that is already installed — it replaces it. A ComfyUI user reported on 2026-09-10 that installing this wheel into a working portable build rewrote PyTorch 2.11+cu130 down to 2.10 and left the launcher failing to start. There is also no yue2, yue2-infer or yue2_infer package on PyPI, so anything answering to that name from an index is something else.

python3 -m venv .yue2 && source .yue2/bin/activate
python -m pip install -U pip huggingface-hub==0.36.2
hf download m-a-p/YuE2-3B yue2_infer-0.1.5-py3-none-any.whl --local-dir .
python -m pip install ./yue2_infer-0.1.5-py3-none-any.whl

Those middle three lines are the model card's quick start unchanged, and 0.1.5 is the version the card names — the same wheel the measurements below were taken on (sha256 8801e2c0…, 66,117 bytes). The repository tree has moved on to 0.1.6. That does not affect anything on this page: unpacking the wheel and comparing it file by file against the GitHub source this recipe links shows 12 of its 14 modules byte-identical, with the differences confined to the version string and to cli.py, where they are a --vae help string and three subcommands that shell out to scripts the wheel does not ship. Every engine module quoted below — pipeline.py, cuda_graph.py, sampling.py, protocol.py, storage.py, quantization.py — is identical in both, and the cli.py line quoted in step 2 is on the same line in both. Install PyTorch's CUDA build first if your environment would otherwise resolve a CPU-only wheel.

2. Confirm the card and the dependencies

yue2 doctor

Read dependencies_ready and the cuda array in the JSON it prints. On the measured machine that reports the RTX 5060 Ti, 15.93 GiB and compute capability 12.0. doctor reports environment readiness only, and says so in its own output — "Environment readiness is not quality or real-24GB acceptance." (src/yue2/cli.py L78).

3. Let the loader fetch the weights

The first pipeline call downloads what it needs from m-a-p/YuE2-3B and m-a-p/YuE2-Vae. The loader uses an explicit allow-list (src/yue2/storage.py L33-39) rather than a full clone, so the demo audio and images in those repositories are skipped — that is why the Storage row is 7.79 GB and not the 7.83 GB the two repositories hold in total. Weight files are hash-checked against weights_manifest.json as they load. Loading took 30.40–34.81 s across the four runs measured here, hash verification included.

Running

Load the pipeline once, then generate:

import json
from pathlib import Path
from huggingface_hub import hf_hub_download
from yue2 import YuE2Pipeline

repo = "m-a-p/YuE2-3B"
demo = json.loads(Path(hf_hub_download(repo, "examples/tonight-awake.json")).read_text(encoding="utf-8"))

pipe = YuE2Pipeline.from_pretrained(repo, device="cuda")
song = pipe(style=demo["style"], lyrics=demo["lyrics"], cot="full", seed=demo["seed"])
song.save("song.flac")
song.save_artifacts("outputs/song")  # ABC, tokens, latents, audio and settings

That is the exact call the two shipped-budget rows below were measured on, with the vendor's own example prompt — which ships inside the model repository and is fetched separately from the weights, because the loader's allow-list does not include it. The two context-filled rows drove the same pipeline one stage at a time (plan → generate_semantic → synthesize → decode) purely so the semantic budget could be set to 24576 - len(plan.prefix) exactly; every other sampling parameter was left at its shipped default.

song.flac is the finished stereo song. outputs/song holds the ABC score, the semantic tokens, the acoustic latents and the settings used — edit score.abc and pass it back as abc= to re-render the same song with a revised melody or harmony.

The same modes are available from the shell, which is the easier path when you want to queue several songs:

yue2 generate --cot full --style "Mandarin funk, nu-disco" --lyrics-file lyrics.txt --output outputs/song

cot="full" plans melody and chords and is the default; cot="melody" plans melody only and is what the vendor recommends for covers; cot="off" generates with no symbolic plan and, as the section above explains, is the mode that costs a second KV branch. Do not pass --budget on this card: the default is already clamped to 13.93 GiB here, so a value of 16 changes nothing and a value of 12 or below lowers the ceiling to 10 GiB and halves the VAE decode tile (src/yue2/pipeline.py L148).

Results

All four rows are ours, on this card, in the environment stated at the top of the page. Peaks are whole-device NVML maxima over the whole run; in-process subtracts the 0.76–0.84 GiB the attached display held at idle, and it is the figure the library's own clamp applies to.

• Speed: 201.04 s of audio in 205.23 s of generation in the default cot="full" mode — real-time factor 1.02 — plus 34.81 s to load. cot="off" is faster (144.39 s for 165.40 s of audio, RTF 0.87) because it skips the symbolic planning stage. The two context-filled runs produced 5.00 and 5.76 minutes of audio in 315.53 s and 298.55 s. Each figure is a single cold run timing nothing but the pipeline call; the vendor's published numbers average 32 warm requests per mode, so small differences between these rows are not signal.
• VRAM usage: 9.03 GiB in-process for an ordinary song and 12.64 GiB in the worst case the library's own allocation rule permits — full context on both guidance branches. The closest any run came to the ceiling was 1.29 GiB, and none of the four ran out of memory.
• The context-filled rows are the point of this page. The clamp gives this card 13.93 GiB and the vendor reports that "maximum-context testing peaked at" 14.08 GiB on its own hardware — a number that, read as a device peak against a process ceiling, appears to rule out a maximum-length song on 16 GB. It does not: the two figures are measured against different things, and the run that fills the context to the token fits with 1.29 GiB to spare here. The vendor does not define what its maximum-context run was, so this page does not claim to have reproduced it — only to have measured the worst case the shipped code can be asked for.
• What transfers to another 16 GB card and what does not. The memory result is a property of the model and the budget: the KV tensors are sized from config.json and the requested token count, and the ceiling is total - 2 GiB on any card. Those hold on any 16 GiB NVIDIA board with BF16 support. The seconds do not. This is a Blackwell card (sm_120); an Ada board of the same capacity has different clocks and memory bandwidth, and on the surfaces searched for this page nobody has measured YuE2 on one.
• Quality notes: the vendor reports a 6.7316 SongBench average for YuE2 and 6.9632 for its best-of-8 setting across 192 WildSongBench prompts, using the legacy decoder — its own automatic metrics under its own candidate-selection protocol, not an independent evaluation. Pass vae="m-a-p/YuE2-Vae-legacy" to from_pretrained to reproduce that protocol; the default decoder is m-a-p/YuE2-Vae, which is what was measured above.

For comparison, the vendor's own reference card is an RTX 4090 24GB, where it publishes 139.48 LM tokens/s, 214.85 s of audio in 71.04 s and an 11.18 GiB NVML peak in cot="full" mode, headlined as "A 3.6-minute song in 71 seconds on an RTX 4090." (model card). Its method note says "NVML records the full-run GPU peak." — the same instrument used here. Read against that, this card is roughly three times slower per second of audio produced (real-time factor 1.02 against 0.33), and its ordinary-song peak is lower rather than higher. Do not read that second observation as efficiency: the two runs differ in card, in allocator ceiling, and in whether the figure is one run or an average of 32, and none of those can be separated from the other two here.

This measurement is filed at /check/yue2-3b/rtx-5060-ti, and the raw session behind it — both measurement scripts and their output — is published on Hugging Face. A second measurement of this pair would land on the same /check page — contribute yours.

Troubleshooting

The unquantized preset requires CUDA BF16 support

This is the pipeline's only hard hardware gate, and it is a BF16 test rather than an architecture test: the constructor calls torch.cuda.is_bf16_supported() and raises that message when it returns false (src/yue2/pipeline.py L158-160). This card satisfies it, and so does any NVIDIA generation that supports BF16. Nothing else in the shipped package tests compute capability on the default path.

An out-of-memory error with memory free on the card

That is the clamp, not the board. set_per_process_memory_fraction holds this process to 13.93 GiB while nvidia-smi still shows free memory — the two-branch context-filled run above peaked with 2.45 GiB physically free on the card and only 1.29 GiB under its own ceiling. Two things make it worse and both look like helping: passing --budget 12 (or lower) caps the process at 10 GiB and halves the VAE decode tile, and running a second generation inside the same Python process leaves the previous run's allocations resident, because PyTorch's caching allocator does not return them to the driver. On the measured machine a loaded pipeline held 7.92 GiB, still 7.92 GiB after del and gc.collect(), and 1.14 GiB after torch.cuda.empty_cache(). Call torch.cuda.empty_cache() between songs, or use one process per song.

Memory budget must leave room for a 2GiB reserve

The same clamp, on a card too small for it: budget comes out at zero or below when the device has 2 GiB or less. On a 12 GB card the ceiling works out to 10 GiB against a documented ordinary-case peak of 11.18 GiB on the vendor's own hardware, and there is no supported way to raise it — the total - 2 term caps whatever you ask for. This recipe's floor is 16 GB for that reason, not because 12 GB is short of raw capacity.

Raising cfg_scale costs a second KV branch

The model card's tuning table suggests cfg_scale=1.2 to strengthen text guidance, and any guidance other than 1 makes the sampler build a second, unconditional branch whose token count is the leading dimension of the KV cache — 2.625 GiB more at full context, as derived above. On this card it is affordable, and the two-branch case itself is measured: the cot="off" rows above both run two branches, and the context-filled one of them is the largest KV allocation the code permits — so raising cfg_scale cannot push that term past the table above. What is not separable from these numbers is the cost of the second branch on its own, because cot="off" differs from cot="full" in more than its branch count: it skips the ABC planning stage entirely. Change one thing at a time and watch the peak.

Experimental FP8 AR requires CUDA compute capability >=8.9

The package has an opt-in FP8 mode for the AR projections, selected with quantization="fp8". It is the only place in the package that tests compute capability (src/yue2/quantization.py L71-72), and this card's 12.0 satisfies it. Leave it off anyway: the module's own status output calls its quality unvalidated, it is incompatible with the CUDA-graph decode path, and it keeps a second copy of the BF16 weights in host memory. The unquantized model fits, as measured.

The GGUF files on the Hub, and the C++ engine that reads them

A GGUF conversion of this model exists at audio-cpp/Yue2-3B-GGUF, and it is not a llama.cpp or Ollama artifact — its header declares general.architecture = "audiocpp", which is not among llama.cpp's registered architectures. It targets 0xShug0/audio.cpp, a separate ggml-based audio engine. That port landed on the engine's dev branch while this page was being written, on 2026-09-10, and its author reports on the vendor's tracker that "The numbers are a bit noisy because of background workloads" around a peak below 9 GB and a real-time factor of 0.23 to 0.28, driven through that engine's UI on an RTX 5090 (vendor issue #163). Treat none of that as transferable here: it is a different engine running a Q8 quantisation, self-reported by the person who wrote the port, on a card with twice this one's memory, and by his own announcement on the model's Hub page the model is in a branch for community testing and "won’t be included in the prebuilt binaries before merging" (discussion 3). If you want it, build the dev branch and expect to be an early tester. This page documents the vendor's Python runtime, which is what the numbers above were measured on.

Community reports so far

YuE2-3B was published on 2026-09-09. Checked 2026-09-10 at 18:58 UTC: m-a-p/YuE2-3B carries three discussion threads, all opened that day — one asking how to get reliably instrumental output, one announcing a ComfyUI node pack, and one announcing the audio.cpp port above — and m-a-p/YuE2-Vae carries none. The same query against the org's YuE v1 model returns ten threads, so the near-silence is the model's age rather than a broken check. The vendor's GitHub tracker holds ten items created since 2026-09-01, nine of them the vendor's own release pull requests and the tenth that same audio.cpp announcement. On every surface searched, no third-party figure for the vendor's Python runtime names a card — the closest is the audio.cpp author's “cpp 0.3 vs python 0.31. ~14G VRAM”, which names neither the card nor which runtime the memory belongs to — — which is why this page is a measurement rather than a citation, and also why one more run of it is worth more than another page of derivation. The hardware issues you will find in the vendor's tracker by searching for "YuE" — a ROCm report, Apple Silicon requests, GGUF requests — all predate YuE2 and were filed against YuE v1, a different model with a different runtime; do not transfer their answers here. Nothing in the current package mentions ROCm, HIP or Metal, and the vendor documents no AMD path.

If you hit something, please report it via the submission form.