How to prepare reference audio

View as Markdown

The quality of your cloned voice is largely determined by the reference clip you provide. This page covers what makes a good reference, what to avoid, and how to handle common situations.

What makes a good reference clip

10-15 seconds

Long enough to capture the voice’s character; short enough to keep latency low and avoid drift.

Clean, single speaker

Only one voice should be present. Background music, noise, and overlapping speech degrade the clone.

Natural prosody

Use a clip with normal sentence intonation - not whispered, shouted, or read in a monotone.

Studio or quiet room

Low reverb, low background noise, and consistent microphone distance produce the most faithful clones.

PropertyRecommendedNotes
FormatWAVFastest to decode and avoids lossy compression artifacts. MP3, FLAC, OPUS, AAC, MULAW, ALAW, and LINEAR16 are also supported.
Sample rate16,000-24,000 HzThe model resamples internally; higher rates do not improve cloning quality but increase upload size.
ChannelsMonoStereo files are downmixed; sending mono saves bandwidth.
Bit depth16-bit PCMSufficient for speech; higher bit depths offer no advantage.

Documented limits

These limits are enforced by the API and a request that exceeds any of them is rejected with a 400.

FieldLimit
text length1000 characters
ref_audio file size10 MB
ref_audio duration30 seconds
ref_text length500 characters

The model is tuned for short reference clips. Even though 30 seconds is accepted, 10-15 seconds is still the sweet spot for clone quality - longer references rarely help and can introduce drift.

What to avoid

Background music or sound effects

Music behind speech is the single most common cause of poor clones. The model can’t separate the voice from the music and may produce muffled or distorted output. Strip music before sending the reference.

Multiple speakers

If two people speak in the reference, the cloned voice will be a blend of both - usually neither sounds correct. Trim the clip down to a single speaker.

Heavy reverb or echo

Recordings made in large rooms, bathrooms, or via speakerphone carry room acoustics that get baked into the clone. The output will sound like it was recorded in the same room - even when you don’t want that.

Heavily compressed or low-bitrate audio

Phone recordings (8 kHz) or low-bitrate MP3 (< 64 kbps) lose the high-frequency detail needed for a faithful clone. Use the highest-quality source available.

Whispered, shouted, or singing voice

The model clones the timbre and prosody of the reference clip. If the reference is whispered, the clone will whisper. Use a reference that matches the speaking style you want in the output.

Providing the reference transcript

You can pass the transcript of the reference audio via the ref_text field. When provided, the model uses it to ground the voice characteristics more precisely.

curl -s -X POST "https://api.sarvam.ai/voices/clone" \
-H "api-subscription-key: $SARVAM_API_KEY" \
-F "text=यह वह वाक्य है जो मैं संश्लेषित करना चाहता हूँ।" \
-F "language_code=hi-IN" \
-F "ref_audio=@reference.wav" \
-F "ref_text=यह संदर्भ ऑडियो की प्रतिलिपि है।"

If you omit ref_text, the API transcribes the reference clip automatically before cloning. Providing the actual transcript usually improves fidelity, especially for short clips or non-English references, and skips the transcription step. Keep it aligned with the spoken content of ref_audio and within 500 characters.

Cross-lingual references

The reference clip’s language does not need to match language_code. You can clone an English voice and have it speak Hindi - the model preserves the speaker’s timbre and identity while applying the phonetics and prosody of the target language. See Clone Across Languages.

Next steps