Self-Hosted DeploymentsOverview

How it works

View as Markdown

A self-hosted deployment has three pieces: an AWS Marketplace subscription (entitlement), the Sarvam model package (the pre-built container Sarvam publishes), and a SageMaker endpoint you create from it inside your own account.

  • The model package is a versioned container Sarvam maintains. You never build or patch it — you subscribe to a version and deploy it.
  • The endpoint runs on GPU instances you choose, in your region and VPC. It is network-isolated: no outbound internet from the container.
  • Your data (audio, documents) is sent only to your own endpoint and stays in your cloud.

Deployment modes

Real-time

An always-on endpoint that returns results in the API response. For interactive and synchronous workloads.

Async

Submit a job, get results from S3 when ready. Supports large payloads and scale-to-zero when idle.

Batch transform

Process a whole S3 prefix of files in one job, then shut the instances down. For bulk/offline processing.

Not every model supports every mode:

  • Saaras v3 (STT): real-time (audio ≤ 30 s) and streaming (bidirectional SigV4 HTTP/2, for longer or continuous audio). Batch is not supported.
  • Bulbul v3 (TTS): real-time, server-side streaming (SSE), bidirectional (SigV4 HTTP/2), and batch transform.
  • Sarvam Vision: real-time (sync), async, and batch transform.

Which mode should I use?

Your workloadUse
Live captions, interactive transcription, short audio clipsReal-time (Saaras v3)
Long or continuous audioStreaming (Saaras v3)
Short text prompts, interactive speechReal-time (Bulbul v3)
Low-latency playback or long text (≤ 3500 chars)Streaming — SSE or bidirectional (Bulbul v3)
A small document that finishes within 60 s, interactivelyReal-time (Vision)
Large documents, occasional or bursty traffic, cost-sensitiveAsync (Vision — scale-to-zero)
Bulk/offline processing of many documents in S3Batch transform (Vision)

For Saaras v3, the real-time path accepts audio up to 30 seconds; for longer or continuous audio use the streaming endpoint. See Deploy Speech-to-Text.