VoxCPM on RTX 4060: Zero-Shot Voice Cloning TTS in ~5 GB VRAM

What You'll Build

A local text-to-speech pipeline using OpenBMB's VoxCPM-0.5B — a 0.5B-parameter TTS model built on the MiniCPM-4 backbone that does zero-shot voice cloning from a short reference clip. You'll generate 16 kHz audio in Chinese or English, either from text alone or by cloning a voice you supply as a few-second .wav.

Hardware data: RTX 4060 (8 GB VRAM) · VoxCPM-0.5B fits in ~5 GB leaving ~3 GB headroom · See benchmark data

ℹ️ Recipe targets VoxCPM-0.5B (legacy), not VoxCPM1.5 or VoxCPM2. The OpenBMB repo now ships three versions: VoxCPM2 (2B, ~8 GB, 48 kHz, 30 languages), VoxCPM1.5 (0.6B, ~6 GB, 44.1 kHz), and VoxCPM-0.5B (0.5B, ~5 GB, 16 kHz). On an 8 GB card the 0.5B version is the most comfortable fit — VoxCPM2's ~8 GB sits right at the ceiling per the official comparison table. Load by passing "openbmb/VoxCPM-0.5B" explicitly to from_pretrained so the helper doesn't default to a newer, larger checkpoint.

Requirements

Component	Minimum	Tested
GPU	6 GB VRAM (model needs ~5 GB per official comparison table)	RTX 4060 (8 GB)
RAM	8 GB	—
Storage	~2 GB (model weights + denoiser/ASR helpers)	—
Software	Python >= 3.10 (<3.13), PyTorch >= 2.5.0, CUDA >= 12.0 (source)	—

Installation

1. Install the voxcpm package

The simplest path is the published PyPI package:

pip install voxcpm

This is the canonical install per both the Hugging Face model card and the official OpenBMB/VoxCPM GitHub README.

2. (Optional) Install from source for the web demo

If you want the Gradio playground, clone the repo and install in editable mode:

git clone https://github.com/OpenBMB/VoxCPM.git
cd VoxCPM
pip install -e .

3. (Optional) Pre-download model weights

Weights download on first inference automatically, but you can pre-fetch them to control where they land:

from huggingface_hub import snapshot_download
snapshot_download("openbmb/VoxCPM-0.5B")

Running

Python — basic synthesis

import soundfile as sf
from voxcpm import VoxCPM

model = VoxCPM.from_pretrained("openbmb/VoxCPM-0.5B")
wav = model.generate(
    text="VoxCPM is an innovative end-to-end TTS model.",
    prompt_wav_path=None,
    cfg_value=2.0,
    inference_timesteps=10,
    normalize=True,
    denoise=True,
)
sf.write("output.wav", wav, 16000)

Note the 16000 sample rate — VoxCPM-0.5B emits 16 kHz audio. Higher inference_timesteps trades speed for quality; lower trades the opposite. Passing the full "openbmb/VoxCPM-0.5B" repo path keeps you on the legacy 0.5B checkpoint that comfortably fits 8 GB; omitting it can resolve to a newer variant.

Python — zero-shot voice cloning

Supply a short reference clip plus its transcript and the model will mimic the speaker's timbre, accent, and pacing:

wav = model.generate(
    text="Hello — this is your cloned voice speaking.",
    prompt_wav_path="reference.wav",
    prompt_text="reference transcript matching the wav",
    cfg_value=2.0,
    inference_timesteps=10,
    denoise=True,
)
sf.write("cloned.wav", wav, 16000)

Gradio web demo

If you installed from source (step 2), launch the local UI:

python app.py

Results

• Speed: RTF (Real-Time Factor) as low as 0.17 on a consumer-grade NVIDIA RTX 4090 — i.e. 1 s of audio synthesised in ~0.17 s of GPU time — per the official OpenBMB README and HF model card. The RTX 4060 has roughly a third of the RTX 4090's memory bandwidth and FP16 throughput, so expect notably slower wall-clock; no RTX 4060 community benchmark has landed yet. Track /check/voxcpm/rtx-4060 for when one does.
• VRAM usage: ~5 GB per the official model comparison table. That leaves ~3 GB headroom on the RTX 4060's 8 GB — enough for the denoiser/ASR helper models to live alongside without offloading.
• Quality notes: 16 kHz output sample rate, 12.5 Hz acoustic frame rate, 2 supported languages (Chinese, English) per the official Models & Versions table. Voice cloning is "continuation-only" — the model continues from your reference clip rather than fully replacing the speaker's identity from scratch. Trained on a 1.8 M-hour bilingual corpus per the HF model card. Apache-2.0 license — commercial use is permitted.

For the full benchmark data, see /check/voxcpm/rtx-4060.

Troubleshooting

torch.compile errors on launch

The VoxCPM helper enables torch.compile optimisations by default. If you hit a torch.compile platform error — common on Windows or older CUDA stacks — pass optimize=False to VoxCPM.from_pretrained so the model loads without graph compilation.

Strained or weird-sounding voice

Per the Hugging Face card, lower the cfg_value (e.g. from 2.0 toward 1.5) to relax adherence to the text; raise it for tighter prompt following. For long or expressive inputs the model may exhibit instability — chunk the text or reduce inference_timesteps if needed.

Background noise in cloned voices

Set denoise=True (the default in the snippets above) or enable "Prompt Speech Enhancement" in the Gradio UI — this pipes the reference clip through iic/speech_zipenhancer_ans_multiloss_16k_base before cloning. Documented on the HF model card.

Accidentally pulled VoxCPM2 or VoxCPM1.5 weights

If you typed VoxCPM.from_pretrained() without an explicit repo argument, or used "openbmb/VoxCPM" (the umbrella repo), the helper may resolve to the latest variant — VoxCPM2 (2B) sits at ~8 GB peak per the official comparison table, which sits at the 4060's 8 GB ceiling with no headroom for other processes. Always pass "openbmb/VoxCPM-0.5B" verbatim and verify the downloaded checkpoint folder name matches.