LAION · speech-to-speech research demo
Humaneness Voice Converter
One source recording, one target voice. Humaneness Voice Converter is our recommended Chatterbox-based pathway for expressive voice conversion. Compare it first with Best-of-16, then Vanilla Chatterbox and three other converters. This refreshed listening set has 40 curated sources (19 real recordings; 19 English and 21 German) and two fixed target voices.
Model and tested inference code · Individual MP3 files · Manifest and per-case scores
Results at a glance
This refreshed set samples 40 clips across 34 emotion labels, with more real speech and stronger German examples than the first edition. Humaneness Voice Converter scores higher than Vanilla Chatterbox on Content Enjoyment (6.453 vs 6.169) and Production Quality (8.009 vs 7.766). Compare speaker match and emotion scores below; these axes can trade off. Listen to decide which balance suits your use.
Higher CE, PQ and target-speaker cosine are preferable; lower emotion MSE is preferable. CE/PQ/emotion are Humaneness Ears Medium estimates; target-speaker cosine comes from CAMPPlus.
| Converter / selection | CE ↑ | PQ ↑ | Emotion MSE ↓ | Target cosine ↑ |
|---|---|---|---|---|
| Humaneness Voice Converter | 6.453 | 8.009 | 0.092 | 0.773 |
| HVC Best-of-16 | 6.460 | 8.009 | 0.072 | 0.795 |
| Vanilla Chatterbox | 6.169 | 7.766 | 0.113 | 0.821 |
| Seed-VC v1 | 5.926 | 7.513 | 0.223 | 0.873 |
| Seed-VC v2 | 5.830 | 7.405 | 0.370 | 0.793 |
| X-VC (2026) | 5.766 | 7.365 | 0.216 | 0.725 |
| HVC Best-of-4 | 6.455 | 8.012 | 0.075 | 0.786 |
| HVC Best-of-8 | 6.459 | 8.011 | 0.073 | 0.791 |
Best-of-N ranks takes by an equal-third standardized combination of CE, speaker cosine and negative emotion MSE, calibrated on 93 other sources. The nested selection reward necessarily increases as N grows; check each individual metric rather than assuming every axis improves. The new curation was decided from source text and source-quality checks, without looking at converter outputs.
Best-of-N: stronger selection, practical batching
For each source/target pair we generated 16 distinct stochastic takes and selected the highest-reward take from nested pools of 1, 4, 8 and 16. The sequential evaluation spent 0.69, 2.54, 4.99 and 9.87 s per pair for N=1/4/8/16, including candidate scoring. The selected reward changes from 0.382 to 0.450; speaker cosine changes from 0.773 to 0.795, and emotion MSE from 0.092 to 0.072. CE/PQ can move differently from the combined reward.
True GPU batching can generate the candidates in one shared forward pass. For a separate 6.53-s benchmark source on one GH200, median warmed decoder+HiFT times were:
| Precision / candidate pool | N=1 | N=4 | N=8 | N=16 |
|---|---|---|---|---|
| FP32 batch | 0.468 s | 0.526 s | 0.831 s | 1.534 s |
| BF16 batch | 0.558 s | 0.605 s | 0.643 s | 0.665 s |
Three warmed repeats, fixed 10 flow steps, cached source tokens/reference/scores; model load, tokenization, Ears/CAMPPlus ranking, MP3 and network excluded. FP32 was faster for this single/4-take short case; BF16 won for 8/16 takes. A Best-of-N winner still waits for candidate scoring and audio preparation. In completed CLI smokes on that clip, four-take generation took 0.616 s plus 0.135 s ranking and 6.351 s preparation; a 16-take batch took 0.752 s generation plus 0.261 s ranking and 6.283 s preparation. See the full method and all three clip lengths.
The upstream Chatterbox VC recipe reports separate RTX 3090 measurements for Vanilla S3Gen; those cannot be substituted for our GH200 Humaneness Voice Converter measurements.
Target voices
Target A is German (CC0); target B is English (CC BY 4.0). A target voice is an acoustic example, not proof of identity or consent.
Evaluation method, source rights and limitations
Official Seed-VC v1/v2 and X-VC checkpoints were run on the same source files and target references, with their official/offline inference paths. The comparisons use one generation per pair for each external arm. Humaneness Voice Converter and Best-of-N use the released full-flow checkpoint. Each displayed audio file is 24-kHz mono, 128-kb/s MP3 in the linked dataset. For per-case provenance, seed, proxy metric and model details see the manifest and model card. The selected source clips are marked CC BY 4.0. External models keep their respective licenses; their weights/code are not hosted here. These are model-estimated proxy scores, not human MOS or an independent emotion judge. 10 source UIDs overlap known RL source UIDs; speaker-level disjointness was not established. This is not a WER benchmark, blinded preference trial or verified speaker identity test. The source curation retains 34 distinct emotion labels, rather than forcing one weak example into every class.
Research preview. Obtain consent and authorization for target voices; do not use generated speech for impersonation or deception.