Your Voice Clone Sounds Robotic — It's the Reference Audio, Not the Model

Your model is reading 6 to 15 seconds of your reference clip and discarding the rest — and it always takes them from the start. XTTS-v2 uses the first 12 seconds for prosody and the first 10 for the decoder. Chatterbox uses the first 10 seconds for its vocoder conditioning and the first 6 for its token prompt. F5-TTS clips references to roughly 12 seconds. IndexTTS-2 cuts at exactly 15. If your clip opens with a breath, a room tone, or a flat "okay, testing one two", that is the voice you cloned.

This is why cycling XTTS → Chatterbox → F5 → IndexTTS never fixes anything. You are changing the model while feeding all four the same bad ten seconds.

Everything below is read out of each project's source and docs, with the constants quoted, so you can check it against your own installed version.

---

How Much of Your Clip Is Actually Used

Every engine truncates, none of them tells you, and the cut is always from the beginning of the file. Verified against each project's source on August 18, 2026:

Three details inside that table are worth stating plainly:

XTTS has two different sets of defaults, and which one you get depends on how you call it. Through the normal TTS API the config defaults apply: gpt_cond_len=12, gpt_cond_chunk_len=4, max_ref_len=10, sound_norm_refs=False. Call get_conditioning_latents() directly and you get that function's own signature defaults instead — 6, 6 and 30 seconds. This is a real source of "it sounded better in the Gradio demo than in my script".

Chatterbox is the one partial exception. Its speaker-verification embedding is computed over the whole 16 kHz reference, while the T3 token prompt is sliced to 6 seconds and the S3Gen decoder reference to 10. So longer clips do contribute a little to voice identity in Chatterbox — but nothing past 10 seconds touches prosody or the vocoder.

F5-TTS is the one that documents it. Its inference README says outright: "Use reference audio <12s and leave proper silence space (e.g. 1s) at the end. Otherwise there is a risk of truncating in the middle of word, leading to suboptimal generation." It also notes the 30-second single-generation budget is the total of prompt plus output. Feed it 25 seconds of reference and you have almost nothing left for speech.

IndexTTS-2 warns you, but only if you ask. The truncation message — Audio too long (N samples), truncating to M samples — prints only when verbose=True. Turn that on the first time you use a new reference.

---

Symptom to Cause

Diagnose in this order — it is roughly the order of how often each one is the real problem.

---

Cause 1: You Cloned the Wrong 10 Seconds

This is the single most common cause, and the fix is a one-line ffmpeg command.

People record a minute of speech, assume the model averages over all of it, and never listen back to just the opening. But that opening is often the worst part of the take: a breath, room tone, a level "so, this is a test of the voice cloning", the point before the speaker has warmed up. The engine copies pitch range, speaking rate and energy from that window and applies it to every sentence you generate.

First, see what you actually have:

ffprobe -v error -show_entries stream=codec_name,sample_rate,channels \
  -show_entries format=duration -of default=noprint_wrappers=1 reference.wav

Then listen to only the part that matters — for XTTS and Chatterbox, the first 10 seconds:

ffmpeg -i reference.wav -t 10 -c copy first10.wav

If first10.wav sounds flat, your clone will sound flat. Cut a better window instead of re-recording:

# take 12 seconds starting at 00:00:23 — the part where you actually sound like yourself
ffmpeg -i reference.wav -ss 23 -t 12 -ac 1 -ar 24000 -c:a pcm_s16le ref_clean.wav

Pick a window with sentence variety. A statement, a question and something with a bit of emphasis in it will give you a wider pitch range than twelve seconds of level narration. Do not pick a window that starts mid-word, and leave roughly a second of quiet at the end — F5-TTS explicitly asks for this, and it does no harm to the others.

---

Cause 2: Noise Floor and Reverb

A steady background hum becomes part of the cloned voice timbre. Reverb is worse, because it cannot be undone.

These models learn "what this speaker sounds like" from a mel spectrogram of your clip. They do not distinguish your voice from the fan behind you — both are in the spectrum, so both get cloned. The audible result is what people describe as buzzy, gritty or thin.

Measure your noise floor before doing anything else:

ffmpeg -i reference.wav -af volumedetect -f null - 2>&1 | grep -E "mean_volume|max_volume"

How to read it: max_volume should sit somewhere around -3 to -1 dB and mean_volume around -20 to -26 dB for speech. A gap of less than about 20 dB between them usually means the recording is either noisy or already compressed and clipped. A max_volume of exactly 0.0 dB means clipping — see the next section.

Gentle broadband denoise, if you need it:

ffmpeg -i reference.wav -af "afftdn=nf=-25" -ac 1 -ar 24000 ref_denoised.wav

Be conservative. afftdn at aggressive settings strips high-frequency detail from the voice too, which trades a buzzy clone for a dull one. If -25 audibly hurts the speech, the recording needs redoing, not rescuing.

Reverb has no ffmpeg fix. Room reflections are convolved with the speech; there is no filter that reliably reverses that. What to check instead: clap once in the room and listen for a tail. If you can hear the room, move the mic within about 20 cm of your mouth, record facing soft furnishings rather than a bare wall, and avoid kitchens, bathrooms and empty rooms. A phone held close in a carpeted bedroom beats a good microphone across an office.

---

Cause 3: Levels and Clipping

XTTS does not normalize your reference by default — sound_norm_refs is False. Neither does it trim silence: librosa_trim_db defaults to None. So a quiet, silence-padded recording goes in exactly as recorded, and the leading silence eats into your 10- or 12-second budget.

Two fixes, in order of preference.

Normalize to a broadcast-style target with EBU R128 loudness normalization:

ffmpeg -i reference.wav -af "loudnorm=I=-18:TP=-2:LRA=11" \
  -ar 24000 -ac 1 -c:a pcm_s16le ref_normalized.wav

Strip leading silence so your window starts on speech:

ffmpeg -i reference.wav \
  -af "silenceremove=start_periods=1:start_duration=0:start_threshold=-50dB" \
  -ac 1 -ar 24000 ref_trimmed.wav

Or turn XTTS's own normalization on and let it handle levels:

tts.tts_to_file(
    text="The line you want spoken.",
    speaker_wav="ref_clean.wav",
    language="en",
    sound_norm_refs=True,     # off by default
    file_path="out.wav",
)

Clipping cannot be normalized away. If max_volume reads 0.0 dB, the waveform is already flat-topped and the distortion is permanent — that is the harsh crackle you hear on loud syllables in the clone. Re-record with input gain 6-10 dB lower.

---

Cause 4: Codec Damage

A Zoom recording, a Discord clip or a 96 kbps MP3 carries codec artifacts straight into the clone. Voice-conferencing codecs are optimized for intelligibility at low bitrate, not for spectral fidelity: they apply aggressive noise suppression, gating and band limiting. What survives is speech you can understand and a spectrum the cloning model reads as a different, thinner voice.

Symptoms: smeared consonants, sibilance that sounds like a lisp, a voice that is recognisably "close but processed".

There is no repair for this. Convert to WAV so you at least stop adding generations of loss:

ffmpeg -i reference.m4a -ac 1 -ar 24000 -c:a pcm_s16le reference.wav

Then, if the source was a call recording, get a local recording of the same speaker instead. Any phone's voice-memo app records at a far higher bitrate than a video call transmits.

---

Cause 5: The Reference Has No Emotional Register

With XTTS-v2, Chatterbox and F5-TTS, one clip carries both identity and delivery. A calm reference gives you a calm clone, permanently. No prompt wording changes this — the prosody comes from the audio.

Two ways out.

Give the engine an expressive window. Re-cut so the used seconds contain the range you want: varied pitch, a question, some emphasis. This is free and it is usually enough.

Or use an engine that separates the two. IndexTTS-2 and IndexTTS-2.5 take a separate emotional reference:

tts.infer(
    spk_audio_prompt="speaker_ref.wav",     # who it sounds like
    emo_audio_prompt="emo_sad.wav",         # how it is delivered
    emo_alpha=0.9,                          # how strongly
    text="The line you want spoken.",
    output_path="gen.wav",
    verbose=True,
)

Or skip the emotional clip entirely and pass an 8-float vector, in the documented order [happy, angry, sad, afraid, disgusted, melancholic, surprised, calm]:

tts.infer(
    spk_audio_prompt="speaker_ref.wav",
    emo_vector=[0, 0, 0.8, 0, 0, 0, 0, 0],  # 0.8 sad
    use_random=False,
    text="The line you want spoken.",
    output_path="gen.wav",
)

The project notes that enabling use_random reduces cloning fidelity, so leave it False when voice match matters. IndexTTS (index-tts/index-tts) is the only one of the four engines here that treats emotion as a separate input.

Chatterbox has a lighter version of the same idea — exaggeration and cfg_weight — covered below.

---

The 10-Minute Reference Rebuild

Do this once and you will stop blaming models. Target: 12-15 seconds, mono, 24 kHz, WAV, starting on speech, ending with about a second of quiet.

1. Record close. Mic 15-20 cm from your mouth, slightly off-axis. Carpeted room, soft furnishings, no bare walls. Kill fans and air conditioning.
2. Say something with shape. Not "testing one two three". Read two or three sentences that include a question and a phrase with emphasis — the delivery in these seconds is the delivery you get back.
3. Record 30-40 seconds so you have material to choose from.
4. Cut the best 12-15 seconds, starting on a word, ending on a completed sentence plus ~1 s of quiet:

   ffmpeg -i raw.wav -ss 8 -t 13 -ac 1 -ar 24000 -c:a pcm_s16le ref.wav
   

5. Normalize:

   ffmpeg -i ref.wav -af "loudnorm=I=-18:TP=-2:LRA=11" -ar 24000 -ac 1 ref_final.wav
   

6. Verify:

   ffmpeg -i ref_final.wav -af volumedetect -f null - 2>&1 | grep -E "mean_volume|max_volume"
   

7. Listen to ref_final.wav end to end. If it does not sound like the voice you want out, nothing downstream will fix that.

For XTTS specifically, do step 4 three times on different sentences and pass all three:

tts.tts_to_file(
    text="The line you want spoken.",
    speaker_wav=["ref_a.wav", "ref_b.wav", "ref_c.wav"],
    language="en",
    file_path="out.wav",
)

XTTS computes a speaker embedding per file and averages them, and concatenates the audio for the GPT conditioning — so multiple short clips are genuinely better than one long take. The other three engines take a single path, so for those you pick your best window.

---

Per-Engine Knobs Worth Knowing

These are the settings that interact with the reference — not general quality dials.

XTTS-v2

tts.tts_to_file(
    text="...",
    speaker_wav="ref_final.wav",
    language="en",
    gpt_cond_len=12,        # seconds used for prosody (config default 12)
    gpt_cond_chunk_len=4,   # chunk size; latents averaged across chunks
    max_ref_len=10,         # seconds used for the decoder (config default 10)
    sound_norm_refs=True,   # default is False
    file_path="out.wav",
)

Raising gpt_cond_len only helps if the extra seconds are good — it is not a quality dial, it is a "use more of my clip" dial. There is a hard floor in the code too: MIN_AUDIO_SECONDS = 0.33, and a shorter chunk raises "Provided reference audio too short". Practically, anything under a few seconds is not worth trying. Setup details are in our XTTS-v2 voice cloning guide, and the wider model background sits on our Coqui TTS model page.

One housekeeping note: the original coqui-ai/TTS repo has had no commits since August 2024, following Coqui's shutdown. The maintained fork is idiap/coqui-ai-TTS, and it is what pip install coqui-tts gives you. If you are on a 2024-era install, some of the defaults above will differ.

Chatterbox

Defaults are exaggeration=0.5, cfg_weight=0.5. From the project's own tips:

• If the reference speaker talks fast, lower cfg_weight to about 0.3 to improve pacing.
• For dramatic delivery, lower cfg_weight to ~0.3 and raise exaggeration to 0.7+. Higher exaggeration speeds speech up, so the lower cfg_weight compensates.
• If an accent bleeds in, that is the reference clip's language not matching your language tag. The documented mitigation is setting cfg_weight to 0.

The repo's own example filename is your_10s_ref_clip.wav, which tells you what the authors expect. Note also that every Chatterbox output carries Resemble AI's Perth neural watermark by default — worth knowing before you publish anything. Install steps are in our Chatterbox TTS setup guide.

F5-TTS

• Keep the reference under 12 s with ~1 s of trailing silence.
• ref_text is the transcription of your reference. Leave it as "" and F5 transcribes it automatically — convenient, but a wrong auto-transcription degrades output, so type it yourself for anything that matters. If you need accurate transcripts of longer material, faster-whisper does it locally.
• Remember the 30-second budget covers prompt plus output for a single generation. Full walkthrough in our F5-TTS setup guide.

IndexTTS-2 / 2.5

• Reference and emotion prompt are both cut at 15 s.
• Run with verbose=True at least once per new reference so you see the truncation warning.
• emo_alpha around 0.6 or lower is the project's recommendation when deriving emotion from the text itself.

If you are still deciding which engine to commit to, our Kokoro vs XTTS vs Chatterbox comparison and the five-model voice cloning roundup cover that choice. For a tone-transfer approach rather than full cloning, see the OpenVoice v2 guide.

---

What We Did Not Test

We are not publishing A/B audio pairs or similarity scores, because we did not run a controlled listening experiment. Doing that honestly means one consented speaker, one script, and reference clips varied one factor at a time — length, sample rate, reverb, noise floor, delivery — pushed through all four engines with a speaker-similarity metric and a UTMOS-style quality score. We have not done that, so this page does not pretend otherwise.

What this page is built on: the truncation constants, defaults and documented guidance in each project's own source and docs, quoted above and linked below. Those are checkable facts, and they are enough to explain most robotic-sounding clones without any listening test at all.

Where we are giving judgement rather than fact, it is labelled: the 12-15 second target, the -18 LUFS loudness figure and the 20 dB noise-floor heuristic are conventional audio-production practice applied to these truncation limits, not measured optima. If you run the experiment properly on your own voice, your numbers beat ours.

---

A Word on Consent

Clone your own voice, or a voice whose owner has explicitly agreed. Reference clips scraped from podcasts, YouTube or client calls are somebody else's identity, and in a growing number of jurisdictions using them is a legal problem as well as an ethical one. Chatterbox watermarks every output it generates; the other engines here do not, which puts the responsibility entirely on you.

---

Sources

• idiap/coqui-ai-TTS — TTS/tts/models/xtts.py and TTS/tts/configs/xtts_config.py for gpt_cond_len, gpt_cond_chunk_len, max_ref_len, sound_norm_refs, librosa_trim_db and the multi-reference behaviour
• resemble-ai/chatterbox — src/chatterbox/tts.py (ENC_COND_LEN, DEC_COND_LEN), the sample-rate constants in s3tokenizer and s3gen, and the README tips on cfg_weight, exaggeration and Perth watermarking
• SWivid/F5-TTS — src/f5_tts/infer/README.md for the <12 s reference guidance, trailing-silence advice and the 30 s total-generation budget
• index-tts/index-tts — indextts/infer_v2.py (_load_and_cut_audio, 15 s) and the README on emo_audio_prompt, emo_alpha, emo_vector and use_random
• FFmpeg filter documentation — volumedetect, loudnorm, silenceremove, afftdn

Source constants and repository statistics read on August 18, 2026.

---

FAQ