> For clean Markdown of any page, append `.md` to the page URL.
> For a complete documentation index, see https://docs.sarvam.ai/llms.txt.
> For full documentation content in one file, see https://docs.sarvam.ai/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.sarvam.ai/_mcp/server.

# Configure & tune

> Tune a Sarvam SageMaker endpoint with container environment variables — concurrency, default output mode, and GPU behaviour — set at model-creation time.

Most deployments need no tuning — the defaults are production-ready. When you do need to adjust behaviour (for example, to raise concurrency on a larger instance), you set **container environment variables** at model-creation time.

## How configuration works

Pass an `Environment` map when you create the model. SageMaker injects these into the model container, so they apply to whatever endpoint you build from that model.

```python
sm.create_model(
    ModelName=endpoint_name,
    PrimaryContainer={
        "ModelPackageName": model_package_arn,
        "Environment": {
            # container tuning variables go here
        },
    },
    ExecutionRoleArn=role,
    EnableNetworkIsolation=True,
)
```

To change a value later, update the model (or create a new one) and point a new endpoint configuration at it.

## Environment variables

Defaults are auto-derived per instance, so most endpoints need no tuning — override only with a reason. The variables below are for the **Sarvam Vision** container.

| Variable                      | Default             | Purpose                                                                                                 |
| ----------------------------- | ------------------- | ------------------------------------------------------------------------------------------------------- |
| `SM_MAX_CONCURRENT_JOBS`      | auto (per instance) | concurrent documents per instance                                                                       |
| `SM_MAX_INFLIGHT`             | `0` (off)           | sync admission valve — sheds bursts with a retryable `503` instead of queueing them into the 60 s cliff |
| `SM_MAX_PAGES`                | `500`               | max pages per document                                                                                  |
| `SM_MAX_FILE_SIZE_MB`         | `200`               | max file size                                                                                           |
| `SM_JOB_WAIT_TIMEOUT_SECONDS` | `900`               | per-document processing budget                                                                          |
| `SM_DEFAULT_LANGUAGE`         | `hi-IN`             | language when the attribute is omitted                                                                  |
| `SM_DEFAULT_OUTPUT_FORMAT`    | `md`                | output format when the attribute is omitted                                                             |
| `OCR3B_GPU_MEM_UTIL`          | auto                | advanced: vLLM GPU memory fraction                                                                      |

## Sync vs async profiles

The same container serves both modes — it can't detect which it was deployed into, and its auto-derived concurrency targets **throughput**, not AWS's 60-second sync deadline. Pin a profile per mode in the Model `Environment`.

**Sync** — fit every request inside 60 s by capping pages and adding the admission valve:

```bash
# small, single-GPU instance (ml.g6.xlarge / ml.g6e.xlarge)
SM_MAX_CONCURRENT_JOBS=1
SM_MAX_INFLIGHT=2
SM_MAX_PAGES=5            # keep sync documents small so they finish inside 60 s
SM_JOB_WAIT_TIMEOUT_SECONDS=75

# larger / multi-GPU instance (a document fans out across GPUs)
SM_MAX_INFLIGHT=<2 x concurrent docs>   # e.g. 8 on ml.g6e.2xlarge
SM_MAX_PAGES=5
SM_JOB_WAIT_TIMEOUT_SECONDS=75
```

**Async** — long jobs and large payloads; keep the container defaults but raise the wait budget:

```bash
SM_MAX_PAGES=500
SM_JOB_WAIT_TIMEOUT_SECONDS=3300         # inside AWS's async ceiling
```

and set `InvocationTimeoutSeconds=3600` per request for large documents.

> **Note**
>
> Set these in the `Environment` map when you create the Model (above). They enforce the sync-concurrency behaviour described in [Deploy Sarvam Vision](/api/self-hosted/sagemaker/deploy-vision#prefer-async-for-documents). Need a knob that isn't exposed? Reach out at [support@sarvam.ai](mailto:support@sarvam.ai).

## Text-to-Speech (Bulbul) concurrency

The Bulbul v3 container admits work through a fixed door and **sheds excess immediately with `503 service_overloaded`** (plus a body `retry_after`) rather than queueing it. Two ceilings matter:

* **HTTP serve ceiling = `WEB_CONCURRENCY` × `MAX_CONCURRENT_HTTP_REQUESTS`** — concurrent real-time / SSE requests admitted.
* **WebSocket cap = `MAX_CONCURRENT_WS_SESSIONS`** — concurrent bidirectional sessions.

| Variable                       | Shipping default (`ml.g6e.xlarge`) | Purpose                                             |
| ------------------------------ | ---------------------------------- | --------------------------------------------------- |
| `WEB_CONCURRENCY`              | `1`                                | web workers; **the lever for more HTTP throughput** |
| `MAX_CONCURRENT_HTTP_REQUESTS` | `5`                                | concurrent HTTP requests per worker                 |
| `MAX_CONCURRENT_WS_SESSIONS`   | `28`                               | concurrent bidirectional (WebSocket) sessions       |
| `MAX_CONCURRENT_MODEL_CALLS`   | `30`                               | concurrent model calls                              |
| `MAX_MODEL_QUEUE_DEPTH`        | `4`                                | model-call queue depth                              |
| `MAX_SEGMENTS_IN_FLIGHT`       | `2`                                | audio segments in flight                            |

With these defaults the **HTTP serve ceiling is 5** (`WEB_CONCURRENCY × MAX_CONCURRENT_HTTP_REQUESTS`) and the WebSocket cap is \~28. Scale throughput by **adding instances or raising `WEB_CONCURRENCY`** — not by choosing a larger single-GPU instance (`xlarge` ≈ `4xlarge` for throughput; vCPU/RAM don't move the needle).

> **Warning**
>
> On this single-web-worker config, **sustained** overload does not hold latency flat — served `p95` drifts up and throughput falls as the queue accumulates. Provision headroom, back off on `503 service_overloaded`, and raise `WEB_CONCURRENCY` (or add instances) for more sustained HTTP throughput.

## Sizing vs tuning

Tuning changes how a single instance behaves; **sizing and autoscaling** change how many instances you run and how big they are. For throughput and cost, start with [Operations](/api/self-hosted/sagemaker/operations).

> Tune a Sarvam SageMaker endpoint with container environment variables — concurrency, default output mode, and GPU behaviour — set at model-creation time.