Deploy Text-to-Speech (Bulbul v3)

Deploy Bulbul v3 as a SageMaker endpoint in your own account. This guide covers a real-time endpoint end to end with boto3, then server-side streaming (SSE) and bidirectional for lower latency. The full, runnable notebook lives in sarvamai/amazon-sagemaker-examples.

Model identifier. In API requests you pass model: bulbul:v3 — it is the only accepted value (bulbul:v3-beta and v2 are rejected). See the API reference.

Bulbul v3 self-hosted supports real-time (InvokeEndpoint), server-side streaming (SSE) (InvokeEndpointWithResponseStream), bidirectional (InvokeEndpointWithBidirectionalStream), and batch transform. Use SSE or bidirectional for long text or low-latency playback.

Prerequisites

Complete Get started on SageMaker first: subscribe to the Bulbul v3 listing, copy your model package ARN for your region, and have an execution role ARN ready. Read the endpoint’s API reference carefully before you integrate — the request contract is enforced by the container.

Recommended instance

Deploy on ml.g6e.xlarge (1× NVIDIA L40S). Scale concurrency by adding instances or raising WEB_CONCURRENCY — not by choosing a larger single-GPU instance (vCPU/RAM don’t change throughput). See Configure & tune for the serve-ceiling knobs.

1. Create the endpoint

Create endpoint

import base64, json, boto3
sm      = boto3.client("sagemaker")
runtime = boto3.client("sagemaker-runtime")
model_name        = "bulbul-v3"
role              = "arn:aws:iam::<account>:role/<your-sagemaker-execution-role>"
# Copy this from your Marketplace subscription, for your region:
model_package_arn = "<Bulbul v3 model-package ARN for your region>"
sm.create_model(
    ModelName=model_name,
    PrimaryContainer={"ModelPackageName": model_package_arn},
    ExecutionRoleArn=role,
    EnableNetworkIsolation=True,
)
sm.create_endpoint_config(
    EndpointConfigName=model_name,
    ProductionVariants=[{
        "VariantName": "AllTraffic",
        "ModelName": model_name,
        "InstanceType": "ml.g6e.xlarge",
        "InitialInstanceCount": 1,
    }],
)
sm.create_endpoint(EndpointName=model_name, EndpointConfigName=model_name)
sm.get_waiter("endpoint_in_service").wait(EndpointName=model_name)

2. Invoke it

Real-time — JSON in, base64 audio out:

Real-time

payload = {
    "text": "Namaste! Welcome to Sarvam.",
    "model": "bulbul:v3",
    "speaker": "shubh",
    "language_code": "en-IN",
    "output_audio_codec": "wav",
    "speech_sample_rate": 24000,
}
r = runtime.invoke_endpoint(
    EndpointName=model_name,
    ContentType="application/json",
    Accept="application/json",
    Body=json.dumps(payload),
)
audio = base64.b64decode(json.loads(r["Body"].read())["audios"][0])

Server-side streaming (SSE) — note "stream": True in the body (the flag lives in the payload, not an API parameter):

Streaming (SSE)

r = runtime.invoke_endpoint_with_response_stream(
    EndpointName=model_name,
    ContentType="application/json",
    Body=json.dumps({**payload, "output_audio_codec": "mp3", "stream": True}),
)
chunks = [e["PayloadPart"]["Bytes"] for e in r["Body"] if e.get("PayloadPart")]

Bidirectional — see the repo notebook text-to-speech/bulbul-v3/bulbul-v3-text-to-speech-Model.ipynb for the full HTTP/2 SigV4 client (aws-sdk-sagemaker-runtime-http2), including the config → text → flush frame loop.

See the full request and response schema in the Text-to-Speech API reference.

Clean up

GPU endpoints bill by the hour. Delete resources you’re not using:

Cleanup

sm.delete_endpoint(EndpointName=model_name)
sm.delete_endpoint_config(EndpointConfigName=model_name)
sm.delete_model(ModelName=model_name)

Prefer infrastructure-as-code?

Deploy the same endpoint with Terraform — IAM role, model, endpoint config, and endpoint in one apply.