Deploy Sarvam Vision

Deploy Sarvam Vision — a 3B-parameter document-intelligence model — as a SageMaker endpoint in your own account. It performs OCR and document parsing (tables, layout, reading order) across 23 languages and returns clean HTML or Markdown. The full, runnable notebook lives in sarvamai/amazon-sagemaker-examples.

Prerequisites

Complete Get started on SageMaker: subscribe to the Sarvam Vision listing, copy your model package ARN, and have an execution role ARN ready.

1. Configure

1
import boto3
2
3
region = boto3.Session().region_name
4
role   = "arn:aws:iam::<account>:role/<your-sagemaker-execution-role>"
5
model_package_arn = "arn:aws:sagemaker:<region>:<vendor>:model-package/<vision-package-id>"
6
7
realtime_instance = "ml.g6e.xlarge"   # real-time / async (recommended default: ~2 concurrent docs)
8
batch_instance    = "ml.g6.xlarge"    # batch
9
endpoint_name     = "sarvam-vision"
10
11
sm      = boto3.client("sagemaker", region_name=region)
12
runtime = boto3.client("sagemaker-runtime", region_name=region)

2. Deploy a real-time endpoint

1
sm.create_model(
2
    ModelName=endpoint_name,
3
    PrimaryContainer={"ModelPackageName": model_package_arn},
4
    ExecutionRoleArn=role,
5
    EnableNetworkIsolation=True,
6
)

1
sm.create_endpoint_config(
2
    EndpointConfigName=endpoint_name,
3
    ProductionVariants=[{
4
        "VariantName": "AllTraffic",
5
        "ModelName": endpoint_name,
6
        "InstanceType": realtime_instance,
7
        "InitialInstanceCount": 2,   # >= 2 for production (survives an instance/AZ loss)
8
        # the ~6 GB model loads in 60-120 s; give startup room or the endpoint fails
9
        "ContainerStartupHealthCheckTimeoutInSeconds": 900,
10
        "ModelDataDownloadTimeoutInSeconds": 1200,
11
    }],
12
)
13
sm.create_endpoint(EndpointName=endpoint_name, EndpointConfigName=endpoint_name)
14
sm.get_waiter("endpoint_in_service").wait(EndpointName=endpoint_name)

3. Invoke it

Send a single PDF, individual PNG/JPG page images, or a flat ZIP of page images as the request body, with a Content-Type that matches the bytes. Pass options — language, output_format (md/html/json), filename — as custom attributes. The full request and response contract is in the API reference.

1
import json
2
3
with open("invoice.pdf", "rb") as f:
4
    resp = runtime.invoke_endpoint(
5
        EndpointName=endpoint_name,
6
        ContentType="application/pdf",                      # must match the bytes
7
        Accept="application/json",
8
        CustomAttributes="language=hi-IN,output_format=md,filename=invoice.pdf",
9
        Body=f.read(),
10
    )
11
12
body = json.loads(resp["Body"].read())
13
print(body["result"]["text"])   # merged document text, structure preserved

Concurrency

Because each sync request must finish inside the 60-second timeout, an instance serves a bounded number of documents at once. That capacity scales with the ml.g6e instance size — a document fans out across the instance’s GPUs. See the recommended instances and concurrency table for the per-instance numbers.

Send more concurrent documents than the instance can handle and requests queue past 60 seconds and time out. For heavy, bursty, or high-concurrency workloads, size up the ml.g6e instance or — better — use async (below), which queues work and never hits the sync timeout.

Prefer async for documents

Async is the recommended path for almost all document workloads. It queues requests, supports payloads up to 50 MB and minutes-long processing, scales to zero when idle, and never hits the 60-second sync timeout.

1
sm.create_endpoint_config(
2
    EndpointConfigName=endpoint_name,
3
    ProductionVariants=[{
4
        "VariantName": "AllTraffic", "ModelName": endpoint_name,
5
        "InstanceType": realtime_instance, "InitialInstanceCount": 1,
6
    }],
7
    AsyncInferenceConfig={
8
        "OutputConfig": {"S3OutputPath": "s3://<your-bucket>/vision/out/"},
9
    },
10
)

• Async — submit a job and read the result from S3 when ready; supports large documents, high concurrency, and scale-to-zero.
• Batch transform — process a whole S3 prefix of documents in one job, then shut the instances down.

Reserve sync for small, latency-sensitive, one-off documents at low concurrency; send everything else to async or batch. Both use the same model package — see Operations for autoscaling and sizing.

Clean up

1
sm.delete_endpoint(EndpointName=endpoint_name)
2
sm.delete_endpoint_config(EndpointConfigName=endpoint_name)
3
sm.delete_model(ModelName=endpoint_name)