Error handling on SageMaker

Every Sarvam model container on SageMaker uses the same error contract. Read this once, then see the per-model references — Speech-to-Text, Text-to-Speech, and Sarvam Vision — for the model-specific bodies.

Every non-2xx is delivered as HTTP 424

Through sagemaker-runtime:InvokeEndpoint, any error the container returns is collapsed by boto3 into a ModelError (HTTP 424). The container’s real status is in OriginalStatusCode, its JSON body in OriginalMessage, and its Retry-After header is dropped — so retry hints ride in the body as retry_after.

1
from botocore.exceptions import ClientError
2
import json
3
4
try:
5
    r = client.invoke_endpoint(EndpointName=EP, ContentType=ct, Body=body)
6
except ClientError as e:
7
    status = e.response["OriginalStatusCode"]           # 400 / 429 / 503 / …
8
    err    = json.loads(e.response["OriginalMessage"])["error"]
9
    if status == 400:
10
        raise                                            # bad request — fix it, never retry
11
    elif err["code"] == "service_overloaded":
12
        backoff(err.get("retry_after", 1))               # 429 or 503 — back off with jitter
13
    else:
14
        retry_once()                                     # 5xx — retry once, then report err["request_id"]

Error envelope

Every error body is JSON with the same shape:

1
{ "error": { "code": "…", "message": "…", "request_id": "…" } }

Optional fields extend it: detail_code (a specific reason), retryable (bool), and retry_after (seconds). Sarvam Vision always carries detail_code and retryable.

Overload (backpressure) sheds immediately — it does not queue

At capacity, the container rejects excess work at once rather than queueing it, using error.code = "service_overloaded" with a body retry_after: