RVCBench: Benchmarking the Robustness of Voice Cloning Across Modern Audio Generation Models

Q: What four dimensions of robustness does RVCBench test?

Input Robustness (reference-audio and text-prompt shifts), Generation Robustness (model architecture, multilingual, long-form, and expressive generalisation), Output Robustness (post-processing resilience and deepfake detectability), and Audio Perturbation Robustness (passive noise, proactive protection methods, and counteract/denoising attacks).

Q: What is RVCBench?

RVCBench is a benchmark for voice-cloning robustness, speaker privacy, and audio-protection methods. It evaluates 32 zero-shot and fine-tuning TTS/VC models against 5 audio-protection methods across 10 dataset configurations, scoring speaker similarity, intelligibility, perceptual quality, and runtime.

Q: How many voice-cloning models does RVCBench evaluate?

The RVCBench codebase includes wrappers for 32 TTS/VC adversary models. The arXiv v2 paper reports results for 18 of those models across 18 robustness evaluations, 225 speakers, and 14,370 utterances.

Q: What audio-protection methods does RVCBench compare?

Five methods on equal footing: SafeSpeech (adversarial perturbation against a surrogate VC model), Enkidu (perceptual-loss adversarial perturbation), EM (Expectation–Maximisation perturbation), Spectral (SafeSpeech's spectral perturbation mode), and GR-Noise (Gaussian random noise).

Q: Which model is hardest to clone under protection, according to RVCBench?

Across the LibriTTS leaderboard, StyleTTS 2 and OpenVoice V2 have the lowest clean speaker similarity and drop furthest under protection — GR-Noise pushes StyleTTS 2's similarity from 0.228 down to 0.030.

Q: Is the RVCBench dataset public?

Yes. The benchmark dataset is hosted on Hugging Face at huggingface.co/datasets/Nanboy/RVCBench under a CC0-1.0 license, with 10 dataset configurations spanning English, Mandarin, and French.

Q: How do I cite RVCBench?

Cite the arXiv preprint: Jin, Ruinan; Liao, Xinting; Yu, Hanlin; Pandya, Deval; Li, Xiaoxiao. “RVCBench: Benchmarking the Robustness of Voice Cloning Across Modern Audio Generation Models.” arXiv:2602.00443, 2026.

Ruinan Jin; Xinting Liao; Hanlin Yu; Deval Pandya; Xiaoxiao Li

Voice-cloning robustness benchmark

32 models will try to clone this voice. 5 defenses try to stop them.

RVCBench is a unified, reproducible pipeline for evaluating voice-cloning robustness, speaker privacy, and audio-protection methods — spanning 32 TTS/VC adversary models, 10 dataset conditions, and 5 protection algorithms, scored on speaker similarity, intelligibility, perceptual quality, and runtime.

See the leaderboard Get the dataset ↗ Try the demo ↗

arXiv:2602.00443 · CC0-1.0 · Jin, Liao, Yu, Pandya, Li

signal.monitor — source → protection → clonelive

■ clean signal (SIM ↑) ■ after protection (SIM ↓)

32

VC / TTS models

5

protection methods

10

dataset configs

225

speakers (paper)

14,370

utterances (paper)

3

languages — EN / ZH / FR

Paper vs. codebase. The arXiv v2 paper reports 18 models across 18 robustness evaluations, 225 speakers, and 14,370 utterances. This repository is under active development and currently ships 32 models; treat any count beyond the paper's 18 as codebase-only until covered by a future revision.

Why RVCBench

One surrogate model is not a threat model.

Most protection papers evaluate against one or two VC models on one dataset. A defense that looks strong there can fail completely against a model or language it was never tested on.

	Typical single-paper eval	RVCBench
Adversary models	1–3	26, zero-shot + fine-tuning
Datasets / languages	1	10, incl. ZH / FR / bilingual / noisy
Protection methods compared	usually own only	5, equal footing
Denoising-adaptive attacker	rarely modeled	built into the pipeline
Metrics	ad hoc	standardised + bootstrap CIs
Reproducibility	custom scripts	one Hydra pipeline, public HF data

“If you're deciding whether a protection method actually generalises — or whether a new TTS model breaks existing defenses — this is the benchmark to run it against.”

How it works

Source audio → protect → (denoise) → clone → evaluate.

Every run walks the same five-stage path, so protection methods and cloning adversaries are always compared on equal footing.

1

Source audio

Reference speaker prompt drawn from one of 10 benchmark dataset configs.

run_vc.py

2

Protection

An adversarial perturbation degrades the prompt: SafeSpeech, Enkidu, EM, Spectral, or Gaussian noise.

run_protect.py

3

Denoise (optional)

An adaptive attacker tries to clean the protected prompt before cloning — a realistic worst case.

run_denoiser.py

4

Voice cloning

One of 32 zero-shot or fine-tuning TTS/VC adversaries attempts to reproduce the speaker.

run_vc_protect.py

5

Evaluation

SIM, WER, MOS, MCD, RTF, SVA and emotion match, with bootstrap confidence intervals.

metrics.json

The framework

Four dimensions, one benchmark.

Every result on this page demonstrates one of four robustness axes. This is the map — each card links to the visualization that proves it.

Input Robustness

Does it still work when the reference audio or text prompt isn't clean studio speech?

Reference-audio shifts — accents, ages, multi-speaker clips, café/station/train noise
Text-prompt shifts — unusual, robocall-style, or hallucination-inducing prompts

See it in the cross-dataset heatmap →

Generation Robustness

Does cloning quality hold up across model architectures, languages, and utterance length?

32 adversary models spanning codec-LM, diffusion, and hybrid architectures
Multilingual (EN/ZH/FR), long-form generation, and emotion preservation

See it in the leaderboard →

Output Robustness

Does the cloned output survive real-world post-processing, and can it be told apart from the real speaker?

Post-processing resilience — MP3/AAC/Opus compression, phone-narrowband simulation
Deepfake detectability — ground-truth vs. cloned speech classification

In the codebase (data/compression/, src/datasets/deepfake_preprocess.py) — not yet on the public leaderboard

Audio Perturbation Robustness

Can a protection method actually stop a clone — and survive an attacker trying to denoise it back out?

Passive perturbation — natural multi-speaker interference and environmental noise
Proactive perturbation (5 methods) and counteract perturbation (adaptive denoising)

See it in the protection-robustness chart →

Generation Robustness Leaderboard — LibriTTS, clean prompts

Which model clones a voice best?

Ranked by speaker similarity (SIM) on unprotected prompts, averaged across the full LibriTTS speaker set. Click a column to sort.

#	Model	SIM ↑▾	WER ↓	MOS ↑	MCD ↓	RTF ↓	SVA ↑	Emo ↑
1	Qwen3-TTS	0.614	0.052	4.39	5.79	2.02	0.974	0.731
2	IndexTTS	0.606	0.052	4.06	6.61	2.23	0.972	0.693
3	CosyVoice 2	0.602	0.175	4.39	6.17	4.58	0.974	0.729
4	ZipVoice	0.579	0.053	4.13	7.09	1.46	0.952	0.675
5	MaskGCT	0.570	0.088	3.93	6.91	1.36	0.939	0.682
6	GLM-TTS	0.570	0.087	4.08	6.41	1.74	0.951	0.678
7	F5-TTS	0.559	0.116	3.99	6.96	0.61	0.937	0.676
8	Higgs Audio	0.559	0.250	4.30	6.06	1.42	0.941	0.717
9	MGM-Omni	0.539	0.095	4.28	5.82	0.84	0.933	0.676
10	PlayDiffusion	0.506	0.055	4.15	8.06	0.73	0.936	0.681
11	MOSS-TTSD	0.492	0.383	4.10	7.09	—	0.876	0.667
12	VibeVoice	0.480	0.228	3.83	6.76	1.86	0.852	0.624
13	FishSpeech	0.472	0.166	4.37	6.47	3.61	0.907	0.682
14	XTTS-v2	0.454	0.073	3.81	8.62	0.62	0.908	0.639
15	SparkTTS	0.408	0.326	4.06	5.83	1.56	0.764	0.672
16	OZSpeech	0.388	0.060	3.21	6.87	8.75	0.840	0.636
17	OpenVoice V2	0.244	0.075	4.30	7.06	0.08	0.474	0.601
18	StyleTTS 2	0.228	0.049	4.30	6.81	0.11	0.388	0.589

SIM: speaker cosine similarity · WER: word error rate · MOS: SpeechMOS perceptual score · MCD: mel-cepstral distortion · RTF: real-time factor (<1 = faster than real time) · SVA: speaker-verification accuracy · Emo: emotion match rate. All values on clean prompts.

Audio Perturbation Robustness Protection robustness — SIM, LibriTTS

How far can protection push similarity down?

Each row spans a model's clean SIM down to its lowest SIM under any of the 5 proactive-perturbation methods below — the larger the gap, the more that model's voice can be shielded. The pipeline's optional Denoise step is the counteract-perturbation half of this dimension: an adaptive attacker trying to undo the protection before cloning.

Clean prompt Best-case protection (lowest SIM reached, method labeled)

Show full data — every protection method, every model

Model	Clean	SafeSpeech	Enkidu	Spectral	GR-Noise	EM
Qwen3-TTS	0.614	0.384	0.502	0.363	0.408	0.582
IndexTTS	0.606	0.346	0.475	0.318	0.392	0.572
CosyVoice 2	0.602	0.321	0.447	0.301	0.384	0.549
ZipVoice	0.579	0.287	0.435	0.262	0.258	0.543
MaskGCT	0.570	0.303	0.407	0.281	0.312	0.530
GLM-TTS	0.570	0.330	0.445	0.311	0.388	0.532
F5-TTS	0.559	0.207	0.431	0.176	0.137	0.520
Higgs Audio	0.559	0.264	0.435	0.236	0.272	0.521
MGM-Omni	0.539	0.184	0.316	0.166	0.229	0.491
PlayDiffusion	0.506	0.173	—	0.149	0.162	0.466
MOSS-TTSD	0.492	0.242	0.335	0.216	0.247	0.453
VibeVoice	0.480	0.272	0.367	0.253	0.280	0.442
FishSpeech	0.472	0.238	0.334	0.212	0.235	0.439
XTTS-v2	0.454	0.260	0.308	0.241	0.237	0.414
SparkTTS	0.408	0.129	0.137	0.108	0.062	0.359
OZSpeech	0.388	0.156	0.187	0.147	0.148	0.337
OpenVoice V2	0.244	0.185	0.188	0.180	0.175	0.236
StyleTTS 2	0.228	0.089	0.125	0.081	0.030	0.207

InputGeneration Cross-dataset generalisation — SIM, clean prompts

Does it still work outside LibriTTS?

Speaker similarity across 10 dataset conditions, grouped by which dimension they test: Input Robustness shifts the reference audio or speaker mix, Generation Robustness shifts the language or utterance length. Hover a cell for the exact value.

Model	Baseline	Input Robustness					Generation Robustness
Model	LibriTTS	VCTK	Multi-spk	BG-clean	BG-noise	Halluc.	Long	AISHELL	French	Bilingual
Qwen3-TTS	0.61	0.62	0.49	0.69	0.57	0.52	0.56	0.72	0.54	0.67
IndexTTS	0.61	0.57	0.47	0.59	0.53	0.53	0.78	0.72	0.40	0.67
CosyVoice 2	0.60	0.58	0.45	0.63	0.52	0.52	0.53	0.72	0.38	0.65
ZipVoice	0.58	0.55	0.53	0.62	0.46	0.51	0.73	0.71	0.36	0.32
MaskGCT	0.57	0.56	0.43	0.61	0.49	0.50	0.19	0.67	0.49	—
GLM-TTS	0.57	0.57	0.45	0.62	0.53	0.53	0.76	0.69	0.40	0.66
F5-TTS	0.56	0.54	0.51	0.58	0.41	0.46	0.61	0.70	0.30	0.65
Higgs Audio	0.56	0.52	0.42	0.59	0.42	0.42	0.52	0.58	0.35	0.54
MGM-Omni	0.54	0.45	0.37	0.52	0.33	0.40	0.44	0.71	0.23	0.63
PlayDiffusion	0.51	0.43	0.36	0.43	0.30	0.41	0.64	0.44	0.28	0.47
MOSS-TTSD	0.49	0.44	0.38	0.49	0.49	0.42	0.64	0.44	0.33	0.47
VibeVoice	0.48	0.44	0.35	0.51	0.36	0.41	0.62	0.56	0.34	0.53
FishSpeech	0.47	0.43	0.38	0.49	0.39	0.35	0.57	0.61	0.37	0.57
XTTS-v2	0.45	0.45	0.33	0.55	0.39	0.49	0.61	0.57	0.45	0.51
SparkTTS	0.41	0.53	0.23	0.59	0.33	0.34	0.34	0.57	0.16	0.48
OZSpeech	0.39	0.25	0.27	0.27	0.16	0.28	—	—	0.11	—
OpenVoice V2	0.24	0.39	0.19	0.48	0.36	0.36	0.28	0.43	0.27	0.30
StyleTTS 2	0.23	0.24	0.16	0.20	0.17	0.18	—	—	—	0.21

0.00.78 SIM

Supported adversaries

32 voice-cloning models, one interface.

Zero-shot and fine-tuning TTS/VC wrappers, each launched from its own Conda environment to keep incompatible dependency stacks apart.

On the LibriTTS leaderboard (18) In the codebase, not yet leaderboard-scored (9)

BertVITS2bert

Qwen3-TTSqwen3_tts

Qwen3-Omniqwen3_omni

FireRedTTS-2fireredtts2

VoxCPMvoxcpm

F5-TTSf5_tts

MaskGCTmaskgct

OpenVoice V2openvoice

Coqui XTTS-v2xtts

IndexTTSindex_tts

ZipVoicezipvoice

FishSpeechfishspeech

Fish Audio S2 (in-proc)fishspeech_s2

Fish Audio S2 (server)fish_audio_s2

CosyVoice / 2cosyvoice

Higgs Audiohiggs_audio

Higgs TTS 3higgs_tts_3

SparkTTSsparktts

VALL-Evall_e

StyleTTS 2styletts2

GLM-TTSglm_tts

GlowTTSglowtts

Kimi Audiokimi_audio

MGM-Omnimgm_omni

MOSS TTSDmoss_ttsd

MOSS-TTSmoss_tts

dots.ttsdots_tts

ZONOS2zonos2

PlayDiffusionplaydiffusion

Bark Voice Clonebark_voice_clone

OZSpeechozspeech

VibeVoicevibevoice

Protection methods

Five ways to make a voice harder to clone.

SafeSpeech

Adversarial perturbation optimised against a surrogate VC model.

Enkidu

Perceptual-loss adversarial perturbation.

EM

Expectation–Maximisation perturbation.

GRNoise

Gaussian random noise — no surrogate model required.

Spectral

SafeSpeech's spectral perturbation mode.

Datasets

10 benchmark conditions, publicly hosted.

Every subset is a Hugging Face dataset config under Nanboy/RVCBench, fetched automatically by the data loader.

Config	Language	Typical use
Libritts	EN	English zero-shot VC/TTS benchmark prompts.
VCTK	EN	Multi-speaker English voice cloning.
Multispeaker_libri	EN	Multi-speaker LibriSpeech-style evaluation.
Long_context	EN	Longer-context voice-cloning prompts.
AISHELL1_dev	ZH	Mandarin speech evaluation.
CommonVoiceFR_dev	FR	French speech evaluation.
Bilingual_uedin	EN/ZH	Bilingual speech evaluation.
Background_noise	EN	Noisy-prompt robustness.
robotcall	EN	Robocall-style speech robustness.
vctk_text_robust	EN	Text robustness on VCTK-style prompts.

Evaluated metrics

The referee's scorecard.

SIM↑

Speaker cosine similarity between cloned and target voice.

WER↓

Word error rate — how intelligible the generated speech is.

MOS↑

SpeechMOS perceptual quality score.

MCD↓

Mel-cepstral distortion versus the reference.

RTF↓

Real-time factor; below 1 is faster than real time.

SVA↑

Speaker-verification accuracy against the target identity.

Emo↑

Emotion match rate between clone and target.

SNR / STOI / DNSMOS

Fidelity metrics reported for protection and denoising runs.

FAQ

Questions people ask about RVCBench.

What four dimensions of robustness does RVCBench test?

Input Robustness (reference-audio and text-prompt shifts), Generation Robustness (model architecture, multilingual, long-form, and expressive generalisation), Output Robustness (post-processing resilience and deepfake detectability), and Audio Perturbation Robustness (passive noise, proactive protection methods, and counteract/denoising attacks).

What is RVCBench?

RVCBench is a benchmark for voice-cloning robustness, speaker privacy, and audio-protection methods. It evaluates 32 zero-shot and fine-tuning TTS/VC models against 5 audio-protection methods across 10 dataset configurations, scoring speaker similarity, intelligibility, perceptual quality, and runtime.

How many voice-cloning models does RVCBench evaluate?

The RVCBench codebase includes wrappers for 32 TTS/VC adversary models. The arXiv v2 paper reports results for 18 of those models across 18 robustness evaluations, 225 speakers, and 14,370 utterances.

What audio-protection methods does RVCBench compare?

Five methods on equal footing: SafeSpeech (adversarial perturbation against a surrogate VC model), Enkidu (perceptual-loss adversarial perturbation), EM (Expectation–Maximisation perturbation), Spectral (SafeSpeech's spectral perturbation mode), and GR-Noise (Gaussian random noise).

Which model is hardest to clone under protection, according to RVCBench?

Across the LibriTTS leaderboard, StyleTTS 2 and OpenVoice V2 have the lowest clean speaker similarity and drop furthest under protection — GR-Noise pushes StyleTTS 2's similarity from 0.228 down to 0.030.

Is the RVCBench dataset public?

Yes. The benchmark dataset is hosted on Hugging Face at huggingface.co/datasets/Nanboy/RVCBench under a CC0-1.0 license, with 10 dataset configurations spanning English, Mandarin, and French.

How do I cite RVCBench?

Cite the arXiv preprint: Jin, Ruinan; Liao, Xinting; Yu, Hanlin; Pandya, Deval; Li, Xiaoxiao. “RVCBench: Benchmarking the Robustness of Voice Cloning Across Modern Audio Generation Models.” arXiv:2602.00443, 2026.

Getting started

One clone, protected, in under a minute.

All outputs — data, generated audio, metrics — are written inside the repository. No checkpoint bundle required to start.

# clone + environment
git clone https://github.com/Nanboy-Ronan/RVCBench.git
cd RVCBench
conda env create -f envs/qwen3-tts.yml && conda activate qwen3

# zero-shot clone, then protect + clone
python scripts/run_qwen3tts_quickstart.py --max-samples 5
python scripts/run_protect_qwen3tts_quickstart.py --max-samples 5

Full quickstart guide ↗ Notebooks ↗ Model environments ↗

Citation

If RVCBench is useful, cite it.

Contributions are welcome — new protection methods, adversary wrappers, dataset adapters, and evaluation metrics. Open an issue or PR on GitHub.

Open an issue ↗ ★ Star on GitHub

@article{jin2026rvcbench,
  title   = {RVCBench: Benchmarking the Robustness of
             Voice Cloning Across Modern Audio
             Generation Models},
  author  = {Jin, Ruinan and Liao, Xinting and Yu, Hanlin
             and Pandya, Deval and Li, Xiaoxiao},
  journal = {arXiv preprint arXiv:2602.00443},
  year    = {2026}
}