YuE2-3B on RTX 5080: a measured 16GB fit, carried from a slower Blackwell card

What You'll Build

A complete song — vocals and accompaniment, 48 kHz stereo — generated on one 16 GB RTX 5080 from a style prompt and lyrics, together with the editable ABC score the model planned it from, so you can revise the melody or harmony and re-render the same song.

Hardware data: RTX 5080 (16GB VRAM, Blackwell, compute capability 12.0) · 9.03 GiB in-process for an ordinary song and 12.64 GiB at full context, measured by us on a 16 GB card of the same architecture · See benchmark data

ℹ️ The memory result here is measured; nothing on this page is a run on an RTX 5080. The vendor's quick start asks for a 24 GB card and publishes no figure for any 16 GB board, so on 2026-09-10 the site operator ran YuE2-3B four times on an RTX 5060 Ti 16GB — two songs at the shipped semantic budget and two more that fill the model's context to the token, one of those in the mode that also builds the second guidance branch. None of the four ran out of memory. Every number in Results belongs to that card and is labelled as such. What carries to an RTX 5080 is the memory conclusion, because the two tensors that decide it are sized from the model's config.json and from the library's own allocator clamp rather than from the board — and both cards are 16 GB. What does not carry is time: the measured card and this one are the same Blackwell generation, but this one runs its memory on twice the bus, so its seconds would be different and nobody has measured them. If you run this pair, send us your numbers.

⚠️ The weights are non-commercial. YuE2's code is Apache 2.0, but the checkpoints this recipe downloads are CC BY-NC 4.0. The repository's LICENSE scopes that to "the YuE2 checkpoint weights in model.safetensors, or the corresponding" model files. The licence text names the weights and does not address the audio you generate with them — if you intend to release or monetise output, read the full licence rather than this paragraph.

ℹ️ One runtime, and it is a Python wheel. YuE2-3B runs through the vendor's own yue2 package with CUDA. There is no llama.cpp, Ollama or LM Studio path; a GGUF conversion exists for a different engine and its status changed on the day this page was written — see Troubleshooting.

Requirements

The vendor states its hardware line as "Linux · Python 3.10+ · 24GB NVIDIA GPU with BF16 support." on the model card, and as "NVIDIA GPU with BF16 support and 24 GB VRAM." in the GitHub repository. That 24GB is a recommendation, not a floor the software enforces — and it is the reason someone opened a request on a low-VRAM inference project the day the model landed, asking "Any possibility to optimise within 16gb vram?" (Wan2GP issue #2283, 2026-09-10). The answer is that no optimisation is needed, and the next section is the arithmetic behind that, followed by the runs that confirm it.

Why 16 GB holds, in bytes — and why a longer song cannot change it

The library caps the process, not the card. YuE2Pipeline.__init__ reads the device's total memory and sets budget = min((memory_budget_gib - 2) * 2**30, total - 2 * 2**30), then enforces it with set_per_process_memory_fraction (src/yue2/pipeline.py L161-165). memory_budget_gib defaults to 24 whatever card you have, so on any 16 GB board the total - 2 term binds. total is what CUDA reports for your particular board, which is a little under 16 GiB — the card these runs were made on reported 15.93 GiB, putting its ceiling at 13.93 GiB. Read your own total out of yue2 doctor (step 2) and subtract 2 rather than copying that number: boards of the same nominal size do not always report the same total. Two consequences hold on every one of them: an out-of-memory error can arrive with memory physically free on the board, and the figure to compare against the ceiling is the process's own peak — not the device peak, which also contains whatever a display or a monitoring tool is holding and which the clamp never sees.

The KV cache is sized by the budget you ask for, not by how many tokens the model emits. The graph runner preallocates, per layer and once each for keys and values, a tensor shaped (branches, capacity, num_key_value_heads, head_dim) where capacity = max(len(prefix)) + max_tokens (src/yue2/cuda_graph.py L70, L90-L92). From config.json — 28 layers, 8 key/value heads, head dimension 128, dtype bfloat16 — one token on one branch costs 28 × 8 × 128 × 2 bytes × 2 tensors = 114,688 B, i.e. 112 KiB. At the full context that is 2.625 GiB on one branch and 5.25 GiB on two. Not one term in that product is a property of the GPU.

Two is the hard maximum, and so is the context. The sampler builds either [prefix] or [prefix, negative] — one branch at guidance exactly 1, two otherwise, and never more (src/yue2/sampling.py L92); cot="off" takes the two-branch path because its default guidance is 1.01 (src/yue2/protocol.py L106). A request whose prefix plus budget exceeds 24,576 tokens is rejected outright rather than quietly trimmed — "Prefix + requested generation budget exceeds 24576; no implicit truncation" (src/yue2/sampling.py L62-63) — and the context itself cannot be changed: the generation config raises "Require context=24576 and midpoint with positive integer steps" for any other value (src/yue2/protocol.py L54-55).

So the worst case is bounded, and it is the case that was measured:

Against a 16 GiB card that leaves 3.987 GiB for activations, the CUDA context and fragmentation — and the measurement lands where that predicts: on the RTX 5060 Ti 16GB, the full-context two-branch run peaked at 12.64 GiB in-process, 0.627 GiB above the arithmetic floor, with 1.29 GiB still under that card's 13.93 GiB ceiling.

A longer song cannot exceed it. Song length past the context limit is realised as more NAR and VAE chunks, not bigger ones: each chunk's width is computed from the context and the prefix, size = min((context - prefix_tokens - 3) // 2, CONTEXT) (src/yue2/protocol.py L141-145), and the chunks are processed one after another. More audio therefore costs more time, not more memory, and the two tensors that dominate the peak are already at their maximum in the table above.

Installation

1. Install the inference package into a dedicated virtual environment

The venv is not a style preference. The wheel pins all eight of its runtime dependencies with ==, torch==2.10.0 among them, and an == pin does not step aside for a newer version that is already installed — it replaces it. A ComfyUI user reported on 2026-09-10 that installing this wheel into a working portable build rewrote PyTorch 2.11+cu130 down to 2.10 and left the launcher failing to start. There is also no yue2, yue2-infer or yue2_infer package on PyPI, and the vendor's own setup notes say so in as many words: "unverified package with a similar name from PyPI." is the thing not to install (skills/yue2-music/references/models-and-setup.md).

python3 -m venv .yue2 && source .yue2/bin/activate
python -m pip install -U pip huggingface-hub==0.36.2
hf download m-a-p/YuE2-3B yue2_infer-0.1.5-py3-none-any.whl --local-dir .
python -m pip install ./yue2_infer-0.1.5-py3-none-any.whl

Those middle three lines are the model card's quick start unchanged, and 0.1.5 is the version the card names — the same wheel the runs below were taken on (sha256 8801e2c0…, 66,117 bytes). The repository tree has moved on to 0.1.6; unpacking both and comparing them file by file leaves 12 of 14 modules byte-identical, with the differences confined to the version string and to cli.py. Every engine module quoted on this page — pipeline.py, cuda_graph.py, sampling.py, protocol.py, storage.py, quantization.py — is identical in both. Install PyTorch's CUDA build first if your environment would otherwise resolve a CPU-only wheel; the pin resolved to 2.10.0+cu128 on the reference machine, which is the build that carries Blackwell sm_120 kernels and therefore the one this card wants.

2. Confirm the card, the dependencies, and your own ceiling

yue2 doctor

Read dependencies_ready and the cuda array in the JSON it prints. The array carries your board's name, its total memory in GiB and its compute capability — subtract 2 from that total and you have the ceiling the clamp will hold this process to. On an RTX 5080 expect a compute capability of 12.0, which NVIDIA lists for the whole 50-series (CUDA GPUs). doctor reports environment readiness only, and says so in its own output — "Environment readiness is not quality or real-24GB acceptance." (src/yue2/cli.py L78).

3. Let the loader fetch the weights

The first pipeline call downloads what it needs from m-a-p/YuE2-3B and m-a-p/YuE2-Vae. The loader uses an explicit allow-list (src/yue2/storage.py L32-40) rather than a full clone, so the demo audio and images in those repositories are skipped — that is why the Storage row is 7.79 GB and not the 7.83 GB the two repositories hold in total. Weight files are hash-checked against weights_manifest.json as they load. On the reference card that took 30.40–34.81 s across four runs, hash verification included; on a card with more memory bandwidth it will be quicker, and how much quicker is not something this page can tell you.

Running

Load the pipeline once, then generate:

import json
from pathlib import Path
from huggingface_hub import hf_hub_download
from yue2 import YuE2Pipeline

repo = "m-a-p/YuE2-3B"
demo = json.loads(Path(hf_hub_download(repo, "examples/tonight-awake.json")).read_text(encoding="utf-8"))

pipe = YuE2Pipeline.from_pretrained(repo, device="cuda")
song = pipe(style=demo["style"], lyrics=demo["lyrics"], cot="full", seed=demo["seed"])
song.save("song.flac")
song.save_artifacts("outputs/song")  # ABC, tokens, latents, audio and settings

That is the exact call the two shipped-budget runs below were measured on, with the vendor's own example prompt — which ships inside the model repository and is fetched separately from the weights, because the loader's allow-list does not include it. The two context-filled runs drove the same pipeline one stage at a time (plan → generate_semantic → synthesize → decode) purely so the semantic budget could be set to 24576 - len(plan.prefix) exactly; every other sampling parameter was left at its shipped default.

song.flac is the finished stereo song. outputs/song holds the ABC score, the semantic tokens, the acoustic latents and the settings used — edit score.abc and pass it back as abc= to re-render the same song with a revised melody or harmony.

The same modes are available from the shell, which is the easier path when you want to queue several songs:

yue2 generate --cot full --style "Mandarin funk, nu-disco" --lyrics-file lyrics.txt --output outputs/song

cot="full" plans melody and chords and is the default; cot="melody" plans melody only and is what the vendor recommends for covers; cot="off" generates with no symbolic plan and, as the section above explains, is the mode that costs a second KV branch. Do not pass --budget on a 16 GB card: the default is already clamped by the board, so a value of 16 changes nothing and a value of 12 or below lowers the ceiling to 10 GiB and halves the VAE decode tile (src/yue2/pipeline.py L148). The vendor also frames this as a one-at-a-time workload — "One song at a time." — and the batch subcommand queues requests rather than running them concurrently.

Results

Measured on a 16 GB card, and that card is not this one

All four rows below were run by the site operator on 2026-09-10 on an RTX 5060 Ti 16GB (Blackwell, compute capability 12.0, driver 591.86, WSL2 Ubuntu 24.04, yue2_infer 0.1.5). Instrumentation was whole-device NVML sampled at 20 Hz — the same instrument the vendor used, and deliberately not torch.cuda.max_memory_allocated(), which excludes both the CUDA context and the allocator's reserved-but-unused blocks and therefore reads low. Each run was a separate process, because PyTorch's caching allocator returns nothing to the driver on its own: after del and gc.collect() the pipeline still held 7.92 GiB, and only torch.cuda.empty_cache() brought it to 1.14 GiB. That card was not headless — a display held 0.76–0.84 GiB throughout — which is why both a device peak and a peak-minus-idle figure are given, the second being the one the clamp actually applies to. One rig, one operator, one run per configuration.

• VRAM usage: 9.03 GiB in-process for an ordinary song and 12.64 GiB in the worst case the library's own allocation rule permits — full context on both guidance branches. The closest any run came to its ceiling was 1.29 GiB, and none of the four ran out of memory. This is the half of the result that transfers to an RTX 5080: the checkpoint and the KV cache are sized by config.json and by the requested token count, and the ceiling is total - 2 GiB on any board, so a 16 GB RTX 5080 lands on the same arithmetic. Its own peak has never been observed, and a card is entitled to differ by the tens of megabytes that the CUDA context and fragmentation account for.
• Speed: this page states none for the RTX 5080, and the omission is deliberate. Seconds are a property of the silicon, and the two cards differ on the axis that matters most to an autoregressive decode. NVIDIA's own comparison lists both as 16 GB GDDR7, but the RTX 5080 on a 256-bit interface against the measured card's 128-bit (compare specs); board-partner tech specs put the pins at 30 Gbps and 28 Gbps respectively (RTX 5080, RTX 5060 Ti), i.e. 960 GB/s against 448 GB/s, a factor of 2.14. Both are the same architecture at the same compute capability, so the comparison is unusually clean: this is the same generation of silicon with twice the memory system behind it. That makes the measured times an upper bound for an RTX 5080 rather than an estimate of it — a generation-matched card with 2.14× the bandwidth will not be slower — and an upper bound is not a measurement. Send us the real number.
• The context-filled rows are the point of this page. The vendor reports that "maximum-context testing peaked at" 14.08 GiB on its own hardware — a number that, read as a device peak against a 16 GB card's process ceiling, appears to rule out a maximum-length song. It does not: the two figures are measured against different things, and the run that fills the context to the token fit with 1.29 GiB to spare. The vendor does not define what its maximum-context run was, so this page does not claim to have reproduced it — only to have measured the worst case the shipped code can be asked for.
• Quality notes: the vendor reports a 6.7316 SongBench average for YuE2 and 6.9632 for its best-of-8 setting across 192 WildSongBench prompts, using the legacy decoder — its own automatic metrics under its own candidate-selection protocol, not an independent evaluation. Pass vae="m-a-p/YuE2-Vae-legacy" to from_pretrained to reproduce that protocol; the default decoder is m-a-p/YuE2-Vae, which is what was measured above.

The two speed figures that do exist, and neither is for this card

The vendor's reference card is an RTX 4090 24GB, where it publishes 139.48 LM tokens/s, 214.85 s of audio in 71.04 s and an 11.18 GiB NVML peak in cot="full" mode, headlined as "A 3.6-minute song in 71 seconds on an RTX 4090." (model card); its method note says "NVML records the full-run GPU peak.", the same instrument used above. Separately, and this is the only third-party figure for this runtime anywhere, the author of a C++ port reported a real-time factor of 0.30 for his own engine against "python 0.31" for the vendor's Python runtime on the vendor's own example prompt (audio.cpp issue #499, 2026-09-10) — with no card named in that comment, no mode, no memory figure attributable to the Python side and no methodology, in a thread where his other numbers are on an RTX 5090. Read those two as bracketing what a large card does, not as anything about a 16 GB one.

Nothing is filed at /check/yue2-3b/rtx-5080 yet, so that page is where the first measurement of this pair will land — contribute yours.

Troubleshooting

The unquantized preset requires CUDA BF16 support

This is the pipeline's only hard hardware gate, and it is a BF16 test rather than an architecture test: the constructor calls torch.cuda.is_bf16_supported() and raises that message when it returns false (src/yue2/pipeline.py L158-160). An RTX 5080 satisfies it, and so does any NVIDIA generation that supports BF16. Nothing else in the shipped package tests compute capability on the default path.

An out-of-memory error with memory free on the card

That is the clamp, not the board. set_per_process_memory_fraction holds the process to total - 2 GiB — 13.93 GiB on the card these runs were made on — while nvidia-smi still shows free memory; its two-branch context-filled run peaked with 2.45 GiB physically free and only 1.29 GiB under its own ceiling. Two things make it worse and both look like helping: passing --budget 12 (or lower) caps the process at 10 GiB and halves the VAE decode tile, and running a second generation inside the same Python process leaves the previous run's allocations resident, because PyTorch's caching allocator does not return them to the driver. On the reference machine a loaded pipeline held 7.92 GiB, still 7.92 GiB after del and gc.collect(), and 1.14 GiB after torch.cuda.empty_cache(). Call torch.cuda.empty_cache() between songs, or use one process per song.

RuntimeError: USE_FLASH_ATTENTION was not enabled for build. on native Windows

The runs this page rests on were made under WSL2, on Linux wheels, and this failure is what a native Windows install hits instead. The graph runner's flash-attention eligibility test asks whether torch.ops.aten._flash_attention_forward exists and whether its schema carries seqused_k (src/yue2/cuda_graph.py L75-76) — a question about the operator's registration, not about whether the build compiled the kernel. On a wheel built without USE_FLASH_ATTENTION the operator is registered and then throws when called, which is exactly what a community ComfyUI wrapper documents for official Windows wheels: "The op exists and then crashes" (ComfyUI-YuE2 README). The vendor's own escape hatch is a documented backend rather than a patch: backend="torch-eager" (or --backend torch-eager) is accepted at src/yue2/pipeline.py L128-129 and turns off the CUDA-graph path at L250, so the flash selection above never runs. Do not pip install flash-attn to fix it; the package has no such dependency, and the runtime's "FlashAttention" is PyTorch's own kernel. Two caveats on the eager path, and they are the reason this page still documents the default: its per-branch KV cache is allocated at len(ids) + max_tokens (src/yue2/sampling.py L76-79), so the memory bound derived above still holds and cannot be exceeded — but its peak was not measured by anyone, and neither was its speed. Asking for flash explicitly instead fails loudly rather than at the first token: attention_backend="flash" raises "Pinned PyTorch variable-length CUDA FlashAttention is unavailable" (src/yue2/cuda_graph.py L82-83).

Memory budget must leave room for a 2GiB reserve

The same clamp, on a card too small for it: budget comes out at zero or below when the device has 2 GiB or less. It also explains why this recipe's floor is 16 GB and not lower. On a 12 GB card the ceiling works out to 10 GiB — both terms of the min evaluate there, so no --budget value raises it — against the vendor's own 11.18 GiB ordinary-case peak on its 24 GB reference card, with the context not reducible and the two candidate levers (--budget 12 --offload-ar) unmeasured by anyone. That is a card with no documented headroom, which is a different statement from a proven OOM, and it is why 12 GB stays out until somebody measures it.

Raising cfg_scale costs a second KV branch

The model card's tuning table suggests cfg_scale=1.2 to strengthen text guidance, and any guidance other than 1 makes the sampler build a second, unconditional branch whose token count is the leading dimension of the KV cache — 2.625 GiB more at full context, as derived above. On a 16 GB card it is affordable, and the two-branch case is measured: the cot="off" rows above both run two branches, and the context-filled one of them is the largest KV allocation the code permits, so raising cfg_scale cannot push that term past the table. What is not separable from those numbers is the cost of the second branch on its own, because cot="off" differs from cot="full" in more than its branch count — it skips the ABC planning stage entirely. Change one thing at a time and watch the peak.

Experimental FP8 AR requires CUDA compute capability >=8.9

The package has an opt-in FP8 mode for the AR projections, selected with quantization="fp8". It is the only place in the package that tests compute capability (src/yue2/quantization.py L71-72), and the RTX 5080's 12.0 clears it comfortably. Leave it off anyway: the module's own status output calls its quality unvalidated, it is incompatible with the CUDA-graph decode path, and it keeps a second copy of the BF16 weights in host memory. The unquantized model fits, as measured.

The GGUF files on the Hub, and the C++ engine that reads them

A GGUF conversion of this model exists at audio-cpp/Yue2-3B-GGUF, and it is not a llama.cpp or Ollama artifact — its header declares general.architecture = "audiocpp", which is not among llama.cpp's registered architectures. It targets 0xShug0/audio.cpp, a separate ggml-based audio engine. That port landed on the engine's dev branch while this page was being written and was still there when it was re-checked: head 3caeba87, committed 2026-09-10 at 17:09 UTC, 29 yue2 paths including src/models/yue2/ and docs/models/yue2.md. Its author reports a peak below 9 GB and a real-time factor of 0.23 to 0.28 through that engine's UI on an RTX 5090, adding that "The numbers are a bit noisy because of background workloads" (vendor issue #163). Two things to keep straight before that sub-9 GB figure changes your plans: it is a different engine running a Q8 quantisation, self-reported by the person who wrote the port, on a card with twice this one's memory and none of it verified by us; and it is the deliberately smaller of his configurations — in the same thread he says a more aggressive cache would push the real-time factor below 0.2 at higher VRAM. By his own announcement the model sits in a branch for community testing that "won’t be included in the prebuilt binaries before merging" (discussion 3). If you want it, build the dev branch and expect to be an early tester. This page documents the vendor's Python runtime, which is what every number above was measured on.

Community reports so far

YuE2-3B was published on 2026-09-09. Re-checked on 2026-09-10 at 21:00 UTC: m-a-p/YuE2-3B carries three discussion threads, all opened the previous day — a question about getting reliably instrumental output, a ComfyUI node-pack announcement, and the audio.cpp port above — and m-a-p/YuE2-Vae carries none. The same query against the org's YuE v1 model returns ten threads, so the near-silence is the model's age rather than a broken check. A GitHub-wide issue search for YuE2 returns 22 items: the vendor's own release pull requests, three requests to port the model to other engines, and two bugs in a community ComfyUI wrapper. Reading the title and body of all 22, all three discussion threads in full, and the comment threads on the port requests: no RTX 5080 appears, and neither does any other 16 GB card — the only consumer cards named anywhere are the vendor's RTX 4090 and the port author's RTX 5090. The hardware issues you will find in the vendor's tracker by searching for "YuE" — a ROCm report, Apple Silicon requests, GGUF requests — all predate YuE2 and were filed against YuE v1, a different model with a different runtime; do not transfer their answers here. Nothing in the current package mentions ROCm, HIP or Metal, and the vendor documents no AMD path.

If you hit something, please report it via the submission form.