How to prepare reference audio

The quality of your cloned voice is largely determined by the reference clip you provide. This page covers what makes a good reference, what to avoid, and how to handle common situations.

What makes a good reference clip

10-15 seconds

Long enough to capture the voice’s character; short enough to keep latency low and avoid drift.

Clean, single speaker

Only one voice should be present. Background music, noise, and overlapping speech degrade the clone.

The performance you want copied

The clone speaks the way the reference speaks. Pace, energy, and emotion in the clip are what the model copies, so record in the style you want in the output and keep that delivery steady. See The clone copies the performance.

Studio or quiet room

Low reverb, low background noise, and consistent microphone distance produce the most faithful clones.

Recommended technical specs

Documented limits

These limits are enforced by the API and a request that exceeds any of them is rejected with a 400.

The model is tuned for short reference clips. Longer references rarely help and can introduce drift.

The clone copies the performance

The model copies more than the speaker’s voice. It also copies how they perform in the reference: pace, energy, and emotion.

A slow, flat read becomes a slow, flat clone. A quick, expressive read becomes a quick, expressive clone. Record the clip in the speaking style you want the generated speech to use. If you want calm narration, read it that way. If you want a conversational agent, speak the way that agent should sound.

Keep that performance consistent through the whole clip. A take that starts calm and ends rushed or shouted gives the model mixed signals, and later generations can swing between those styles. If you record more than one take, use the same pace and emotion in each one.

Whispering, shouting, and singing are copied the same way. A whispered reference produces a whispered clone.

What to avoid

Background music or sound effects

Music behind speech is the single most common cause of poor clones. The model can’t separate the voice from the music and may produce muffled or distorted output. Strip music before sending the reference.

Multiple speakers

If two people speak in the reference, the cloned voice will be a blend of both - usually neither sounds correct. Trim the clip down to a single speaker.

Heavy reverb or echo

Recordings made in large rooms, bathrooms, or via speakerphone carry room acoustics that get baked into the clone. The output will sound like it was recorded in the same room - even when you don’t want that.

Heavily compressed or low-bitrate audio

Phone recordings (8 kHz) or low-bitrate MP3 (< 64 kbps) lose the high-frequency detail needed for a faithful clone. Use the highest-quality source available.

A delivery you do not want copied

Whispering, shouting, singing, and a pace or emotion that changes mid-clip are all copied into the clone. Pick one performance and hold it for the whole reference.

Create the voice or send the clip inline

Saved voice (reuse): upload the reference clip with create_voice / POST /voices/create. The API transcribes it automatically and returns a voice_id.

from sarvamai import SarvamAI
client = SarvamAI(api_subscription_key="YOUR_SARVAM_API_KEY")
voice = client.voice_cloning.create_voice(
    name="my-voice",
    language="hi-IN",
    file=open("reference.wav", "rb"),
)
# voice.data.voice_id → pass to convert / convert_stream / voice_cloning_streaming

Use the returned data.voice_id with convert / POST /voices/clone, convert_stream / /voices/clone/stream, or WebSocket config.

Inline generate: send the same clip as ref_audio on each generate request (optional ref_text). Limits are stricter (10 MB, 15 seconds). Prefer a saved voice_id when you will call the API repeatedly.

Cross-lingual references

The reference clip’s language does not need to match language_code. You can clone an English voice and have it speak Hindi - the model preserves the speaker’s timbre and identity while applying the phonetics and prosody of the target language. See Clone Across Languages.

Next steps