# RVCBench

> A comprehensive voice cloning evaluation package with automatic speech metrics and datasets. Supports scoring your own audio and evaluating models with versioned benchmark suites.

Install with `pip install "rvcbench[eval]"` and run `rvcbench setup-scorers`. Use `rvcbench.metrics.Evaluator` with your own audio, or `rvcbench prompts` and `rvcbench score` with onboarding-v1 (52 utterances), core-v1 (480), or full-v1 (12,724). Metrics include SIM, SVA, WER, MOS, MCD, STOI and emotion consistency. Dataset scoring supports --resume; comparison validates scoring fingerprints.

## Key facts

### Can I score my own audio without using the datasets?

Yes. Install rvcbench[eval], prepare the scorers with rvcbench setup-scorers, and use rvcbench.metrics.Evaluator. Choose from SIM, SVA, WER, MOS, MCD, STOI and emotion consistency. MCD and STOI require a recording of the same text; other metrics use the reference voice or expected text.

### How do I evaluate a model using RVCBench data?

Run rvcbench prompts to prepare a versioned suite, synthesize the listed texts with your model, and run rvcbench score on its WAV files. Start with onboarding-v1 (52 utterances), then use core-v1 (480) or full-v1 (12,724). Scoring, reports and --resume are included in the pip package. The packaged suites are previews; paper results use a separate protocol.

### What is RVCBench?

RVCBench is a general-purpose package for voice cloning evaluation, with automatic speech metrics and ready-to-use datasets. It provides 32 integration entries and paper results for 22 models (18 in the main results, 4 in the appendix), with 5 audio-protection methods across 10 dataset configurations, scoring speaker similarity, intelligibility, perceptual quality, and runtime.

### How many voice-cloning models does RVCBench evaluate?

The RVCBench codebase includes 32 TTS/VC integration entries. The paper (arXiv v3) reports results for 18 of those models across 18 robustness evaluations, 204 speakers, and 14,370 utterances.

### What audio-protection methods does RVCBench compare?

Five methods on equal footing: SafeSpeech (adversarial perturbation against a surrogate VC model), Enkidu (perceptual-loss adversarial perturbation), POP (error-minimizing perturbation, named em in the code), Spectral (SafeSpeech's spectral perturbation mode), and GR-Noise (Gaussian random noise).

### Which model is hardest to clone under protection, according to RVCBench?

Across the LibriTTS leaderboard, StyleTTS 2 and OpenVoice V2 have the lowest clean speaker similarity and drop furthest under protection — GR-Noise pushes StyleTTS 2's similarity from 0.23 down to 0.03.

### Is the RVCBench dataset public?

Yes. The benchmark dataset is hosted on Hugging Face at huggingface.co/datasets/Nanboy/RVCBench under a CC0-1.0 license, with 10 dataset configurations spanning English, Mandarin, and French.

### How do I cite RVCBench?

Cite the NeurIPS 2026 paper: Jin, Ruinan; Liao, Xinting; Yu, Hanlin; Pandya, Deval; Li, Xiaoxiao. “RVCBench: Benchmarking the Robustness of Voice Cloning Across Modern Audio Generation Models.” Advances in Neural Information Processing Systems (NeurIPS), 2026. arXiv:2602.00443.


## Links

- [Paper (arXiv:2602.00443)](https://arxiv.org/abs/2602.00443)
- [Dataset (Hugging Face)](https://huggingface.co/datasets/Nanboy/RVCBench)
- [Interactive demo](https://huggingface.co/spaces/Nanboy/RVCBench)
- [Code repository](https://github.com/Nanboy-Ronan/RVCBench)
- [Documentation](https://nanboy-ronan.github.io/RVCBench/docs/)
- [Quickstart](https://nanboy-ronan.github.io/RVCBench/docs/quickstart/)
- [Automatic metrics for your audio](https://nanboy-ronan.github.io/RVCBench/docs/metrics/)
- [Evaluate models with benchmark data](https://nanboy-ronan.github.io/RVCBench/docs/adding_a_model/)
- [Evaluation FAQ and coverage](https://nanboy-ronan.github.io/RVCBench/docs/faq/)
- [Python API reference](https://nanboy-ronan.github.io/RVCBench/docs/api/)
- [PyPI package](https://pypi.org/project/rvcbench/)
- [Full documentation text](https://nanboy-ronan.github.io/RVCBench/llms-full.txt)

## Citation

```
@inproceedings{jin2026rvcbench,
  title   = {RVCBench: Benchmarking the Robustness of Voice Cloning Across Modern Audio Generation Models},
  author  = {Ruinan Jin and Xinting Liao and Hanlin Yu and Deval Pandya and Xiaoxiao Li},
  booktitle = {Advances in Neural Information Processing Systems},
  url     = {https://arxiv.org/abs/2602.00443},
  year    = {2026}
}
```