> For clean Markdown of any page, append `.md` to the page URL.
> For a complete documentation index, see https://docs.sarvam.ai/llms.txt.
> For full documentation content in one file, see https://docs.sarvam.ai/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.sarvam.ai/_mcp/server.

# Operations

> Run Sarvam SageMaker endpoints in production: instance sizing and pricing, autoscaling and scale-to-zero, CloudWatch monitoring, security and data residency, and updating model versions.

Everything you need to run a Sarvam endpoint in production.

## Instance sizing & pricing

All three models run best on the **`ml.g6e`** (L40S) family. Start with the recommended instance and scale up only if a single instance can't meet your throughput.

| Model                   | Real-time / streaming | Batch           |
| ----------------------- | --------------------- | --------------- |
| **Saaras v3 (STT)**     | `ml.g6e.xlarge`       | not supported   |
| **Bulbul v3 (TTS)**     | `ml.g6e.xlarge`       | `ml.g6e.xlarge` |
| **Sarvam Vision (OCR)** | `ml.g6e.xlarge`       | `ml.g6.xlarge`  |

Larger and multi-GPU `ml.g6e` instances scale up accordingly. See [Get started](/api/self-hosted/sagemaker/get-started#recommended-instances) for the per-instance concurrency table.

> **Note**
>
> Pricing is the **software fee** shown on the AWS Marketplace listing, billed per host-hour, **plus** the underlying AWS infrastructure cost. Rates vary by region and package version — always confirm the current rate on the [Speech-to-Text](https://aws.amazon.com/marketplace/pp/prodview-6a2ei6d2iimli) or [Sarvam Vision](https://aws.amazon.com/marketplace/pp/prodview-exwi6jgzqsqc2) listing. Usage-based, no end date, cancel anytime.

## Autoscaling

Attach an Application Auto Scaling policy to the endpoint variant to add and remove instances with load — typically targeting `SageMakerVariantInvocationsPerInstance` or GPU utilisation.

> **Tip**
>
> **Async endpoints can scale to zero.** Set the minimum instance count to 0 so you pay nothing between jobs — the endpoint spins an instance up when a request lands. This is the cheapest option for spiky or occasional workloads.

## Monitoring

#### CloudWatch metrics

Track `Invocations`, `ModelLatency`, `OverheadLatency`, `Invocation4XXErrors`/`5XXErrors`, and GPU/host utilisation per variant. Alarm on latency and error rate.

#### Container logs

The model container streams logs to CloudWatch Logs under the endpoint's log group — your first stop for diagnosing failed invocations.

## Security & data residency

* **Network isolation** — deploy with `EnableNetworkIsolation=True`; the container has no outbound internet.
* **Stays in your cloud** — audio and documents are sent only to your endpoint and never leave your VPC.
* **VPC & encryption** — run the endpoint in your VPC, use your KMS keys for S3 (batch/async) and volume encryption, and scope the execution role to only the buckets it needs.

## Updating model versions

#### Subscribe to the new version

New model versions appear on the Marketplace listing. Copy the new package ARN.

#### Create a new model and endpoint config

Create a fresh model from the new ARN and a new endpoint configuration pointing at it.

#### Update the endpoint in place

Call `update_endpoint` with the new config. SageMaker performs a rolling (blue/green) update with no downtime, then retires the old variant.

> **Note**
>
> Remember the split between the **package version** (what you subscribe to, e.g. `saaras:v3.1`) and the **API model id** (what you send in requests, e.g. `saaras:v3`). A version bump changes the package ARN — it does not necessarily change the request-time model id.

> Run Sarvam SageMaker endpoints in production: instance sizing and pricing, autoscaling and scale-to-zero, CloudWatch monitoring, security and data residency, and updating model versions.