Deploy Speech-to-Text (Saaras v3)

Deploy Saaras v3 as a SageMaker endpoint in your own account. This guide covers a real-time endpoint end to end with boto3, then streaming for longer audio. The full, runnable notebook lives in sarvamai/amazon-sagemaker-examples.

Model identifier. In API requests you pass model: saaras:v3. The Marketplace package is versioned saaras:v3.1 — that’s the package revision you subscribe to, not the value you send in a request. See the API reference.

Saaras v3 self-hosted supports real-time (audio up to 30 seconds) and streaming (WebSocket, for longer/continuous audio). Batch transform is not currently supported for Saaras v3 — use the streaming endpoint for long recordings.

Prerequisites

Complete Get started on SageMaker first: subscribe to the listing, copy your model package ARN, and have an execution role ARN ready. Read the endpoint’s API reference carefully before you integrate — the request contract is enforced by the container.

1. Configure

Configuration

1 import boto3
2 
3 region       = boto3.Session().region_name
4 role         = "arn:aws:iam::<account>:role/<your-sagemaker-execution-role>"
5 # Copy this from your Marketplace subscription, for your region:
6 model_package_arn = "arn:aws:sagemaker:<region>:<vendor>:model-package/<saaras-package-id>"
7 
8 realtime_instance = "ml.g6e.xlarge"   # 1x NVIDIA L40S
9 endpoint_name     = "saaras-stt"
10 
11 sm      = boto3.client("sagemaker", region_name=region)
12 runtime = boto3.client("sagemaker-runtime", region_name=region)

2. Deploy a real-time endpoint

Create the model

Point a SageMaker model at the Marketplace package. Keep network isolation on so the container has no internet egress.

1 sm.create_model(
2     ModelName=endpoint_name,
3     PrimaryContainer={"ModelPackageName": model_package_arn},
4     ExecutionRoleArn=role,
5     EnableNetworkIsolation=True,
6 )

Create the endpoint config

1 sm.create_endpoint_config(
2     EndpointConfigName=endpoint_name,
3     ProductionVariants=[{
4         "VariantName": "AllTraffic",
5         "ModelName": endpoint_name,
6         "InstanceType": realtime_instance,
7         "InitialInstanceCount": 1,
8     }],
9 )

Create the endpoint

1 sm.create_endpoint(EndpointName=endpoint_name, EndpointConfigName=endpoint_name)
2 sm.get_waiter("endpoint_in_service").wait(EndpointName=endpoint_name)

Provisioning a GPU endpoint typically takes 10–15 minutes. It’s ready when its status is InService.

3. Invoke it

Send audio as multipart/form-data (the only accepted content type — see the API reference). Choose an output mode with the mode field (transcribe, translate, verbatim, translit, codemix).

Invoke

1 # build_multipart() assembles the form fields + audio file into a multipart body;
2 # see the reference notebook for the full helper.
3 boundary = "----sarvam-boundary"
4 with open("call.wav", "rb") as f:
5     body = build_multipart(
6         fields={"model": "saaras:v3", "mode": "transcribe", "with_timestamps": "true"},
7         file_bytes=f.read(),
8         boundary=boundary,
9     )
10 
11 resp = runtime.invoke_endpoint(
12     EndpointName=endpoint_name,
13     ContentType=f"multipart/form-data; boundary={boundary}",
14     Accept="application/json",
15     Body=body,
16 )
17 print(resp["Body"].read().decode())

See the full request and response schema in the Speech-to-Text API reference.

The real-time REST path accepts audio up to 30 seconds. For longer or continuous audio, use the streaming endpoint below — batch transform is not supported for Saaras v3.

Streaming (live audio)

For continuous, low-latency transcription, deploy the model and invoke the bidirectional streaming operation InvokeEndpointWithBidirectionalStream — a two-way SigV4 HTTP/2 stream on port 8443. The message protocol and parameters (language-code, sample_rate, vad_signals) are documented in the API reference.

Clean up

GPU endpoints bill by the hour. Delete resources you’re not using:

Cleanup

1 sm.delete_endpoint(EndpointName=endpoint_name)
2 sm.delete_endpoint_config(EndpointConfigName=endpoint_name)
3 sm.delete_model(ModelName=endpoint_name)

Prefer infrastructure-as-code?

Deploy the same endpoint with Terraform — IAM role, model, endpoint config, and endpoint in one apply.