Self-Hosted DeploymentsAWS SageMakerDeploy

Deploy Text-to-Speech (Bulbul v3)

View as Markdown

Deploy Bulbul v3 as a SageMaker endpoint in your own account. This guide covers a real-time endpoint end to end with boto3, then server-side streaming (SSE) and bidirectional for lower latency. The full, runnable notebook lives in sarvamai/amazon-sagemaker-examples.

Model identifier. In API requests you pass model: bulbul:v3 — it is the only accepted value (bulbul:v3-beta and v2 are rejected). See the API reference.

Bulbul v3 self-hosted supports real-time (InvokeEndpoint), server-side streaming (SSE) (InvokeEndpointWithResponseStream), bidirectional (InvokeEndpointWithBidirectionalStream), and batch transform. Use SSE or bidirectional for long text or low-latency playback.

Prerequisites

Complete Get started on SageMaker first: subscribe to the Bulbul v3 listing, copy your model package ARN for your region, and have an execution role ARN ready. Read the endpoint’s API reference carefully before you integrate — the request contract is enforced by the container.

Deploy on ml.g6e.xlarge (1× NVIDIA L40S). Scale concurrency by adding instances or raising WEB_CONCURRENCY — not by choosing a larger single-GPU instance (vCPU/RAM don’t change throughput). See Configure & tune for the serve-ceiling knobs.

1. Create the endpoint

Create endpoint
1import base64, json, boto3
2
3sm      = boto3.client("sagemaker")
4runtime = boto3.client("sagemaker-runtime")
5
6model_name        = "bulbul-v3"
7role              = "arn:aws:iam::<account>:role/<your-sagemaker-execution-role>"
8# Copy this from your Marketplace subscription, for your region:
9model_package_arn = "<Bulbul v3 model-package ARN for your region>"
10
11sm.create_model(
12    ModelName=model_name,
13    PrimaryContainer={"ModelPackageName": model_package_arn},
14    ExecutionRoleArn=role,
15    EnableNetworkIsolation=True,
16)
17sm.create_endpoint_config(
18    EndpointConfigName=model_name,
19    ProductionVariants=[{
20        "VariantName": "AllTraffic",
21        "ModelName": model_name,
22        "InstanceType": "ml.g6e.xlarge",
23        "InitialInstanceCount": 1,
24    }],
25)
26sm.create_endpoint(EndpointName=model_name, EndpointConfigName=model_name)
27sm.get_waiter("endpoint_in_service").wait(EndpointName=model_name)

2. Invoke it

Real-time — JSON in, base64 audio out:

Real-time
1payload = {
2    "text": "Namaste! Welcome to Sarvam.",
3    "model": "bulbul:v3",
4    "speaker": "shubh",
5    "language_code": "en-IN",
6    "output_audio_codec": "wav",
7    "speech_sample_rate": 24000,
8}
9r = runtime.invoke_endpoint(
10    EndpointName=model_name,
11    ContentType="application/json",
12    Accept="application/json",
13    Body=json.dumps(payload),
14)
15audio = base64.b64decode(json.loads(r["Body"].read())["audios"][0])

Server-side streaming (SSE) — note "stream": True in the body (the flag lives in the payload, not an API parameter):

Streaming (SSE)
1r = runtime.invoke_endpoint_with_response_stream(
2    EndpointName=model_name,
3    ContentType="application/json",
4    Body=json.dumps({**payload, "output_audio_codec": "mp3", "stream": True}),
5)
6chunks = [e["PayloadPart"]["Bytes"] for e in r["Body"] if e.get("PayloadPart")]

Bidirectional — see the repo notebook text-to-speech/bulbul-v3/bulbul-v3-text-to-speech-Model.ipynb for the full HTTP/2 SigV4 client (aws-sdk-sagemaker-runtime-http2), including the config → text → flush frame loop.

See the full request and response schema in the Text-to-Speech API reference.

Clean up

GPU endpoints bill by the hour. Delete resources you’re not using:

Cleanup
1sm.delete_endpoint(EndpointName=model_name)
2sm.delete_endpoint_config(EndpointConfigName=model_name)
3sm.delete_model(ModelName=model_name)
Prefer infrastructure-as-code?

Deploy the same endpoint with Terraform — IAM role, model, endpoint config, and endpoint in one apply.