WebSocket Streaming API

GET /voices/clone/ws (WebSocket upgrade). Connect once, send a config message with either a saved voice_id or base64 ref_audio (optional ref_text), then stream text and flush. The server returns base64-encoded audio frames and a final event when synthesis completes.

Send exactly one of voice_id or ref_audio in config.data.

The wire protocol matches Bulbul TTS WebSocket, so a client that already speaks that protocol can retarget with a different path and config payload.

Common use cases:

• Conversational agents: Stream TTS responses in real time for voice assistants that use a cloned voice
• Incremental LLM output: Feed tokens as they arrive; flush when the utterance ends
• Multi-turn sessions: Reuse one connection across many utterances

For one-shot generation without a persistent socket, prefer HTTP streaming.

Connection

wss://api.sarvam.ai/voices/clone/ws

Auth: send your key in the api-subscription-key header, or as a WebSocket subprotocol api-subscription-key.<your-key>.

Optional query parameter: send_completion_event=true (default) to receive a final event when synthesis completes.

Message flow

1. Connect and authenticate.
2. Send { "type": "config", "data": { ... } } once. Must include target_language_code and exactly one of voice_id or ref_audio.
3. Send one or more { "type": "text", "data": { "text": "..." } } messages (max 2500 characters each).
4. Send { "type": "flush" } to force remaining buffered text through synthesis.
5. Receive { "type": "audio", "data": { "audio": "<base64>", "content_type": "...", "request_id": "..." } } frames.
6. Receive { "type": "event", "data": { "event_type": "final" } } when the utterance is done.
7. Optionally send { "type": "ping" } to keep long-lived connections alive.

Config fields

Not accepted: min_audio_duration, max_audio_duration, enable_qc, enable_vad, enable_cached_responses.

Code example

Prefer the official SDK (voice_cloning_streaming / voiceCloningStreaming). Socket verbs are configure → convert → flush. In Python ≥ 0.1.37, pass typed message models (see below). See Libraries & SDKs.

import asyncio
import base64
from sarvamai import AsyncSarvamAI, AudioOutput
from sarvamai.types.flush_signal import FlushSignal
from sarvamai.types.voice_cloning_configure_connection import (
    VoiceCloningConfigureConnection,
    VoiceCloningConfigureConnectionData,
)
from sarvamai.types.voice_cloning_send_text import (
    VoiceCloningSendText,
    VoiceCloningSendTextData,
)
async def clone_stream():
    client = AsyncSarvamAI(api_subscription_key="YOUR_SARVAM_API_KEY")
    async with client.voice_cloning_streaming.connect() as ws:
        await ws.configure(
            VoiceCloningConfigureConnection(
                data=VoiceCloningConfigureConnectionData(
                    target_language_code="hi-IN",
                    voice_id="svc-your-voice-id",
                    output_audio_codec="mp3",
                )
            )
        )
        await ws.convert(
            VoiceCloningSendText(
                data=VoiceCloningSendTextData(
                    text="नमस्ते। यह एक वेबसॉकेट स्ट्रीमिंग टेस्ट है।"
                )
            )
        )
        await ws.flush(FlushSignal())
        with open("output.mp3", "wb") as f:
            while True:
                try:
                    message = await asyncio.wait_for(ws.recv(), timeout=3.0)
                except asyncio.TimeoutError:
                    break
                if isinstance(message, AudioOutput):
                    f.write(base64.b64decode(message.data.audio))
                    f.flush()
if __name__ == "__main__":
    asyncio.run(clone_stream())

Full message schemas are on the WebSocket endpoint in the API Reference.

Error handling

• Validation and auth failures before or during the session arrive as { "type": "error", "data": { "message": "...", "code": ... } } frames.
• After the first audio frame, a mid-stream model failure may close the connection; there is no separate binary error envelope.
• Keep the socket alive with periodic ping messages on long-lived sessions.

Next steps