YuE2-3B on RTX 3090: full songs on Ampere without the FP8 path

What You'll Build

A complete song — vocals and accompaniment, 48 kHz stereo — generated on one RTX 3090 from a style prompt and lyrics, together with the editable ABC score the model wrote for it, so you can revise the melody or harmony and re-render the same song.

Hardware data: RTX 3090 (24GB VRAM) · no measurement exists for this card · the fit below is derived from the shipped code · See benchmark data

⚠️ The weights are non-commercial. YuE2's code is Apache 2.0, but the checkpoints this recipe downloads are CC BY-NC 4.0. The repository's LICENSE says the weights are "licensed under Creative Commons Attribution-NonCommercial 4.0 International" and scopes that to "the YuE2 checkpoint weights in model.safetensors, or the corresponding" model files. On GitHub the vendor splits the two explicitly: "YuE2's first-party code, agent skill, and documentation are licensed under" Apache 2.0, while "Model weights are separately licensed under" CC BY-NC 4.0. The licence text names the weights and does not address the audio you generate with them — if you intend to release or monetise output, read the full licence rather than this paragraph.

ℹ️ One runtime, and it is a Python wheel. YuE2-3B runs through the vendor's own yue2 package on Linux with CUDA. There is no llama.cpp, Ollama, LM Studio or ComfyUI path — see Troubleshooting for what the GGUF files on the Hub are and are not.

⚠️ Nobody has run this model on an Ampere card and published the numbers. The vendor's published table covers exactly two GPUs and neither is this one. Everything below about whether it runs is derived from the shipped source, which you can re-check line by line; everything about how fast it runs is missing, and this recipe says so rather than filling the gap.

Requirements

The vendor states the hardware line as "Linux · Python 3.10+ · 24GB NVIDIA GPU with BF16 support." on the model card, and as "Linux · Python 3.12 · NVIDIA GPU with BF16 support and 24 GB VRAM." in the GitHub repository. Both name a capacity and neither names an architecture, a generation or a compute capability — the software floor the shipped code actually enforces is derived in Troubleshooting.

Installation

1. Install the inference package into a dedicated virtual environment

These three lines are the model card's quick start, unchanged:

python3 -m venv .yue2 && source .yue2/bin/activate
python -m pip install huggingface-hub==0.36.2
hf download m-a-p/YuE2-3B yue2_infer-0.1.5-py3-none-any.whl --local-dir .
python -m pip install ./yue2_infer-0.1.5-py3-none-any.whl

Create and activate a fresh virtual environment before running them, because every one of the wheel's eight runtime dependencies is pinned with == rather than >= — torch==2.10.0, transformers==4.57.6, huggingface-hub==0.36.2, safetensors==0.7.0, tiktoken==0.12.0, numpy==2.2.6, soundfile==0.13.1, accelerate==1.13.0, read from the wheel's own METADATA and matching the repository's pyproject.toml. An exact pin does not skip an already-satisfied environment; it rewrites it. That is not hypothetical: a ComfyUI user reported on 2026-09-10 that installing this wheel into a working portable build downgraded PyTorch 2.11 to 2.10 and left the launcher failing to start.

The wheel this recipe installs is 0.1.5, the version the model card's quick start names; the repository tree is 0.1.6, and the source files this recipe cites are byte-identical between the two, so everything below holds for either install path. The pins also match the versions the vendor used for the numbers in Results.

Install PyTorch's CUDA build first if your environment would otherwise resolve a CPU-only wheel.

2. Confirm the card and the dependencies

The package ships a yue2 command whose doctor subcommand prints your card's name, its memory in GiB and its compute capability, alongside the resolved version of every pinned dependency:

yue2 doctor

Read dependencies_ready and the cuda array in the JSON it prints. On this card the compute capability it reports is the number the FP8 section below turns on. doctor reports environment readiness only — the vendor's own note in that output is explicit that it is not an acceptance test.

3. Let the loader fetch the weights

The first pipeline call downloads what it needs from m-a-p/YuE2-3B and m-a-p/YuE2-Vae. The loader uses an explicit allow-list (src/yue2/storage.py L33-39) rather than a full clone, so the demo audio and images in those repositories are not downloaded — that is why the Storage row above is 7.79 GB and not the 7.83 GB the two repositories hold in total.

Running

Load the pipeline once, then generate:

from pathlib import Path
from yue2 import YuE2Pipeline

pipe = YuE2Pipeline.from_pretrained("m-a-p/YuE2-3B", device="cuda")

Then run the vendor's own example prompt, which is shipped inside the model repository:

import json
from huggingface_hub import hf_hub_download

repo = "m-a-p/YuE2-3B"
prompt_path = hf_hub_download(repo, "examples/tonight-awake.json")
demo = json.loads(Path(prompt_path).read_text(encoding="utf-8"))
style, lyrics = demo["style"], demo["lyrics"]

song = pipe(style=style, lyrics=lyrics, cot="full", seed=demo["seed"])
song.save("song.flac")
song.save_artifacts("outputs/song")  # ABC, tokens, latents, audio and settings

song.flac is the finished stereo song. outputs/song holds the ABC score, the semantic tokens, the acoustic latents and the exact settings used — edit score.abc and pass it back as abc= to re-render the same song with a revised melody or harmony.

The same three modes are available from the shell, which is the easier path when you want to queue several songs:

yue2 generate --cot full --style "Mandarin funk, nu-disco" --lyrics-file lyrics.txt --output outputs/song

cot="full" plans melody and chords and is the default; cot="melody" plans melody only and is what the vendor recommends for covers; cot="off" generates with no symbolic plan. The default decoder is m-a-p/YuE2-Vae; pass vae="m-a-p/YuE2-Vae-legacy" to from_pretrained to reproduce the published benchmark protocol instead.

Results

No speed or VRAM figure below was measured on this card. The vendor published one consumer-GPU row and it is a different GPU, and no published community figure names an Ampere card. What is on this page is that row, attributed, plus the parts of it that are card-independent.

• Speed — unmeasured here. The vendor's consumer row reports 139.48 LM tokens/s in cot="full" mode, producing 214.85 s of audio in 71.04 s of wall clock, and its headline for that is "A 3.6-minute song in 71 seconds on an RTX 4090." (model card). The card the vendor measured is a generation newer than this one, so that figure is an optimistic ceiling an RTX 3090 will not reach — not an estimate of what it will do, and not a floor. How far below it this card lands is the number nobody has taken on an Ampere board. Method, verbatim: "HF: PyTorch 2.10, Transformers 4.57.6, no quantization, default YuE2-Vae." and "4090 values average 32 warm requests per mode". The runtime is described as "The HF package uses PyTorch, CUDA graphs, and FlashAttention, with BF16 AR/NAR and FP32 VAE."
• VRAM usage — the fixed part transfers, the variable part does not. The vendor's peaks are 11.18 GiB in full mode, 11.02 GiB in melody and 11.09 GiB in off, all NVML readings — "NVML records the full-run GPU peak." — so they include the CUDA context and not only the allocator. Maximum-context testing is separately reported at 14.08 GiB.
• Which parts of that peak are architecture-independent, and which cannot be separated out at all: two weight figures are exact and are properties of the checkpoint, so they hold wherever you load it — the AR/NAR checkpoint is 6.763 GiB in BF16 and the VAE decoder is 0.494 GiB, byte counts read from the Hugging Face tree API on 2026-09-10 (YuE2-3B, YuE2-Vae). They are not two slices of one peak, and adding them together describes no moment in the run. The pipeline evicts the language model to host RAM before the decoder ever reaches the card: decode() moves the AR/NAR model to the host (src/yue2/pipeline.py L323-324) and empties the CUDA cache (L326) before the VAE is loaded on the CPU and moved onto the device (L331-333), then returns it to the host in the finally (L354); L222 is the only line in the module that ever puts the AR/NAR model on the device. The rest of the peak — KV cache, activations, CUDA-graph pools, CUDA context — is not decomposable from anything the vendor published: no per-component figure exists, and the model card does not say which stage the NVML reading was taken in.
• What the 24GB actually gives you: the library caps the process below the card's capacity, so the headroom that matters is the 22 GiB ceiling derived in Troubleshooting, not the 12.82 GiB of raw capacity left over after an 11.18 GiB peak.
• Throughput: "One song at a time." The pipeline is not a batching server; the batch subcommand queues requests, it does not run them concurrently on this path.
• Quality notes: the vendor reports a 6.7316 SongBench average for YuE2 and 6.9632 for its best-of-8 setting across 192 WildSongBench prompts, using the legacy decoder. Those are the vendor's own automatic metrics under its own candidate-selection protocol, not an independent evaluation, and they are properties of the model rather than of the hardware.

No benchmark exists for this pair. If you run it, please send us the numbers — a measured figure replaces the vendor's on /check/yue2-3b/rtx-3090, which is where the live data for this pair lands.

Troubleshooting

Will an Ampere card run this? Every gate it has to clear, enumerated

The vendor never says. What the shipped package does is checkable, and there are only two hardware tests in it.

The first is the pipeline's own, and it is a BF16 test rather than an architecture test: the constructor calls torch.cuda.is_bf16_supported() and raises The unquantized preset requires CUDA BF16 support when it returns false (src/yue2/pipeline.py L159-160). Ampere supports BF16, so it passes. The second is the FP8 gate in the next section, which the default path never reaches.

Nothing else in the package tests the device's compute capability. In particular the attention selection does not: it picks PyTorch's fused FlashAttention when the dtype and head dimension allow it — head dimension is 128 here, from the model's config.json — and falls back to cuDNN and then to plain SDPA when they do not, with no architecture test anywhere in that chain. There is also no flash-attn package to install: it appears nowhere in the wheel's dependency list, and the runtime says as much in its own comment — "CUDA normally uses PyTorch's fused SDPA without an external flash package."

The one remaining question is whether the pinned torch==2.10.0 was even compiled for this card, and PyTorch's own release script answers it. The base architecture list for a CUDA build is 7.0;7.5;8.0;8.6;9.0, and 8.6 is Ampere's GA102 — the RTX 3090's silicon. The comment that names it is in the ARM filter, which strips 8.6 on aarch64 only: "Remove: < 8.0 (no ARM GPUs), 8.6 (x86_64 RTX 3090/A6000 only)" (.ci/manywheel/build_cuda.sh). On x86_64 that filter never runs, so the kernels for this card are in the wheel — and the reason they are there is, in PyTorch's own words, this card.

Experimental FP8 AR requires CUDA compute capability >=8.9

This is the one message an Ampere owner can hit that an Ada owner cannot, and it is entirely avoidable: it comes from an opt-in mode you have to ask for. quantization="fp8" is the only place in the package that reads compute capability (src/yue2/quantization.py L71-72), and it requires 8.9 or newer, which this card does not have.

Losing it costs nothing measurable, because nothing about it has been measured. The module's own docstring calls it "Opt-in experimental FP8 AR linear layers; NAR always restores exact BF16." and states that "No quantized quality or speed claim is implied by enabling this module.", and the runtime's status output labels its quality and performance validation unvalidated. Every published YuE2-3B number was taken at the default, quantization="none", and the model fits a 24GB card unquantized anyway. Leave the default alone.

Memory budget must leave room for a 2GiB reserve — and the real VRAM floor

The pipeline takes a memory_budget_gib argument that defaults to 24, then clamps it to whatever the card can actually give: it reads the device's total memory and sets the budget to the smaller of memory_budget_gib − 2 and total − 2 GiB, capping the process with set_per_process_memory_fraction (src/yue2/pipeline.py L161-165). The clamp depends on the card's capacity alone, which is why it is the one part of the vendor's setup this recipe can reproduce exactly:

• 24GB card, this one included: the budget works out to about 22 GiB, the two terms landing within rounding of each other on a card this size — so the process ceiling here is the same one the published figures were measured under. The 11.18 GiB ordinary peak and the 14.08 GiB maximum-context peak both clear it with room to spare.
• 16GB card: the budget works out to about 14 GiB, and the whole range fits. Measured here on an RTX 5060 Ti 16GB (2026-09-10, WSL2, driver 591.86): an ordinary song peaked at 10.05 GiB — below the vendor's own 4090 figure — and a deliberately full-context song run with the second CFG branch, the worst case the library can be put in, peaked at 13.48 GiB on the device and 12.64 GiB in-process, leaving 1.29 GiB under the clamp. No OOM in any run. An earlier version of this page said a maximum-context song would not fit 16GB; that was wrong, and wrong in an instructive way — it compared the vendor's 14.08 GiB, which is a device peak, against a ceiling set_per_process_memory_fraction imposes on the process. A device peak includes whatever a monitor is holding; the clamp never sees it.
• 12GB card: the budget works out to about 10 GiB, which is below the 11.18 GiB the vendor measured for the ordinary case. The documented install does not fit a 12GB card, and the failure arrives as an allocator error rather than the message above.

Shrinking the context is not an available workaround for any of this: the generation config rejects any value but 24576 outright (src/yue2/protocol.py L54-55). Pass the budget argument explicitly if the default is wrong for your machine — YuE2Pipeline.from_pretrained("m-a-p/YuE2-3B", device="cuda", memory_budget_gib=16), or --budget 16 on the command line. Note that a budget of 12 or less also halves the VAE core frame count, which is a real change to the decode path and not only a memory setting.

One setting moves memory the other way, and the model card recommends trying it. Its tuning table offers cfg_scale=1.2 to "Experiment with stronger text guidance", and any guidance other than 1 makes the sampler build a second, unconditional branch (src/yue2/sampling.py L92) whose KV cache is allocated with the branch count as its leading dimension (src/yue2/cuda_graph.py L90). So raising it costs additional memory. How much is not published, and it cannot be recovered from the vendor's three-mode table: cot="off" already runs two branches — its default guidance is 1.01 (protocol.py L106) and the branch test is cfg_scale == 1 — yet it is measured at 11.09 GiB, below the 11.18 GiB of cot="full" on one branch. The three modes differ in more than their branch count, so the gaps between them cannot be used to price the second branch. A 24GB board has the headroom to absorb it; a 16GB one may not, and there is no published figure to check against.

The GGUF files on the Hub are not a runtime path

A GGUF conversion of this model exists at audio-cpp/Yue2-3B-GGUF, converted from the same upstream revision this recipe pins. It is not a llama.cpp or Ollama artifact — it targets a separate C++ engine, 0xShug0/audio.cpp, and the GGUF repository's own card says that engine's support for this model is still in development and due to land on the engine's dev branch. That changed hours after this page was first published. The port landed on the engine's dev branch on 2026-09-10 at 17:09 UTC — head 3caeba87, 29 paths including src/models/yue2/, include/engine/models/yue2/ and docs/models/yue2.md — and its author reports a peak below 9 GB and a real-time factor of 0.23 to 0.28 through that engine's UI on an RTX 5090 (vendor issue #163). Treat none of it as transferable here: a different engine running a Q8 quantisation, self-reported by the person who wrote the port, and by his own announcement the model sits in a branch for community testing that “won’t be included in the prebuilt binaries before merging” (discussion 3). If you want it, build the dev branch and expect to be an early tester. This page documents the vendor's Python runtime, which is what every number above was measured on.

No community reports yet

YuE2-3B was published on 2026-09-09. Re-checked on 2026-09-10: m-a-p/YuE2-Vae has no discussion threads and m-a-p/YuE2-3B has one, opened that day — a usage question about getting reliably instrumental output, tried in the hosted Space and naming no GPU. The same query against the org's YuE v1 model returns ten threads, so the near-silence is the model's age and not a broken check. The only thread that reports a run is the port author's, and it is through his own C++ engine on a Q8 quantisation rather than the vendor's Python runtime this page documents — a thread count decays by the hour, that predicate does not. The vendor's GitHub tracker holds only release-engineering issues from the same two days, and a tracker-wide search for YuE2 across GitHub on 2026-09-10 returns nothing but vendor pull requests, requests to port the model to third-party engines, and bugs in a community ComfyUI wrapper — not one of them names an Ampere card or reports a run on one. Every hardware-shaped issue you will find by searching that repository for "YuE" — AMD, macOS and GGUF alike — predates YuE2 and was filed against YuE v1, a different model with a different runtime; do not transfer their answers here. Nothing in the current package mentions ROCm, HIP or Metal, and the vendor documents no AMD or Apple path.

If you hit something, please report it via the submission form.