API Guides & TutorialsLarge Language Models

Chat Completion API

View as Markdown

Use the Chat Completion API to build conversational AI experiences with native support for Indian languages and deep contextual reasoning. Sarvam provides two versions: V1 for Sarvam models and beta V2 for Sarvam and open-weight models.

Use the Sarvam chat skill to generate correct chat completion code from your AI coding assistant:

npx skills add sarvamai/skills --skill chat

See Agent Skills for the full list.

Our Chat Completion APIs support the following chat models:

Sarvam-105B

Model ID sarvam-105b. Flagship model for complex reasoning, coding, and agentic workflows. 128K token context length.

Sarvam-105B Conversations

Model ID sarvam-105b-conversations. Conversational variant post-trained for real-time dialogue and voice-agent workloads.

Simply pass the model name as the model parameter (e.g., model="sarvam-105b" or model="sarvam-105b-conversations").

Endpoints: POST /v1/chat/completions serves sarvam-105b and sarvam-105b-conversations. POST /v2/chat/completions serves sarvam-105b, glm5.3, gemma4, and deepseekv4-flash. See Open-Weight Models.

Token budgeting: the context length covers everything, your messages, any reasoning_content the model produces in think mode, and the generated reply (capped by max_tokens, default 2048). Reasoning tokens are billed as completion tokens, so high reasoning_effort increases both latency and cost. For long conversations, trim or summarize older turns instead of resending the full history.

Authentication: like every Sarvam API, this endpoint uses the api-subscription-key header. It additionally accepts Authorization: Bearer <key> for OpenAI-compatible tooling. See Authentication for details.

Sarvam-M (24B) has been deprecated and is no longer available through the Chat Completions API. Please migrate to Sarvam-105B for improved performance.

V1 Chat Completion

POST /v1/chat/completions supports sarvam-105b and sarvam-105b-conversations. Use V1 for Sarvam’s flagship and conversational models. See the Chat Completion V1 API Reference for the complete request and response schema.

from sarvamai import SarvamAI
client = SarvamAI(api_subscription_key="YOUR_SARVAM_API_KEY")
response = client.chat.completions(
    model="sarvam-105b",
    messages=[{"role": "user", "content": "What is the capital of India?"}],
)
print(response.choices[0].message.content)

V2 Chat Completion (Beta)

POST /v2/chat/completions supports sarvam-105b, glm5.3, gemma4, and deepseekv4-flash. Use V2 for the OpenAI-compatible beta API and open-weight models. See the Chat Completion V2 API Reference for the complete request and response schema.

from sarvamai import SarvamAI
client = SarvamAI(api_subscription_key="YOUR_SARVAM_API_KEY")
response = client.chat.completions_v2(
    model="glm5.3",
    messages=[{"role": "user", "content": "Explain photosynthesis briefly."}],
)
print(response.choices[0].message.content)

Beta access: V2 requires whitelisting per API key. Without access you get 400 invalid_request_error before the model runs. See Beta APIs.

V2 behavior (all models)

  • Model IDs are case-sensitive. An unknown ID returns 404 not_found_error.
  • Auth failures return 403, not 401, with invalid_api_key_error.
  • Structured output is available on both V1 and V2 through response_format; on V2, use client.chat.completions_v2. See the Chat Completion V2 API Reference.
  • Parallel tool calls behave differently by model and route. On Chat Completions, parallel_tool_calls: false is correctly enforced on gemma4 and deepseekv4-flash, but not on glm5.3 or sarvam-105b, those two may still return more than one tool call even with false set. On the Responses API, false isn’t currently enforced for any model that supports tool calling there.
  • Reasoning (glm5.3, deepseekv4-flash) has three modes: low, high, and max. Chat Completions accepts all three directly for both models. On the Responses API, DeepSeek V4 Flash also accepts all three directly, but GLM-5.3 only accepts low or high, to get max-level reasoning from GLM-5.3 there, omit the reasoning field entirely, since the literal string "max" returns 400. See Open-Weight Models, Reasoning.
  • Streaming is optional on both V2 Chat Completions and Responses. Omit stream or set it to false for a single JSON response (the default), or true for server-sent events.
  • Log probabilities (logprobs/top_logprobs) on Chat Completions V2 are supported on sarvam-105b and gemma4. They aren’t supported on glm5.3, which returns a clean 400, or on deepseekv4-flash, which returns a 503 model_overloaded instead of a clean rejection, omit the field for that model.

GLM-5.3 and other open-weight models

Open-weight models on V2 have model-specific limits around context size, n, log probabilities, tool-call flags, and stop-sequence behavior. See GLM-5.3, Known limitations and the other Open-Weight model guides.

V2 limits (summary)

TopicV2 notes
Modelssarvam-105b, glm5.3, gemma4, deepseekv4-flash. Confirm with GET /v2/models
Context / max_tokensPer model (e.g. 1M tokens on glm5.3). Chat’s max_tokens defaults to 2048 there, and reasoning counts against that budget. Responses’ max_output_tokens has no default, omitting it means unbounded generation.
nglm5.3 accepts only 1; higher values return 400
stopUp to 4 sequences; on glm5.3, more than four may return 503 instead of 400
Tool callsChat: false is enforced on gemma4/deepseekv4-flash, not on glm5.3/sarvam-105b. Responses: false isn’t enforced for any model.
Reasoning (glm5.3, deepseekv4-flash)Chat accepts low/high/max directly for both (default max). Responses: DeepSeek V4 Flash also accepts all three; GLM-5.3 accepts only low/high, omit the field for max (the string "max" returns 400).
Log probabilitiesSupported on sarvam-105b and gemma4. Not supported on glm5.3 (400) or deepseekv4-flash (503).
streamOptional on Chat V2 and Responses, the default is a single non-streaming JSON response.
ErrorsInvalid JSON or types return 400 with error.message; overload and some stop-related cases return 503 (retry with backoff).

The Limits table at the bottom of this page applies to V1 (sarvam-105b / sarvam-105b-conversations), not V2 open-weight models.

Features

Hybrid Thinking Mode
  • Supports both “think” and “non-think” modes
  • Think mode for complex logical reasoning
  • Non-think mode for efficient conversations
  • Ideal for mathematical and coding tasks
Advanced Indic Skills
  • Post-trained on Indian languages
  • Native English proficiency
  • Authentic Indian cultural values
  • Rich understanding of local context
Superior Reasoning Capabilities
  • Outperforms similar-sized models
  • Strong performance on coding tasks
  • Excellent mathematical reasoning
  • Advanced problem-solving abilities
Seamless Chatting Experience
  • Full Indic script support
  • Romanized language support
  • Multilingual conversation handling
  • Natural language understanding

V1 features and examples

from sarvamai import SarvamAI
client = SarvamAI(
    api_subscription_key="YOUR_SARVAM_API_KEY",
)
response = client.chat.completions(
    model="sarvam-105b",
    messages=[
        {"role": "user", "content": "Hey, what is the capital of India?"}
    ],
)
print(response)
Key Considerations

  • Reasoning effort options: low, medium, high

    • Thinking mode is on by default (low); pass reasoning_effort=None (Python) / reasoning_effort: null (JS, cURL) to disable it
    • Higher values increase reasoning depth
    • Reasoning tokens (returned as reasoning_content) count toward your completion tokens and bill. Use lower effort or disable reasoning for latency- and cost-sensitive paths

  • Output length is capped by max_tokens (default 2048), raise it for long-form generation

Because thinking mode is on by default, a low max_tokens (e.g. under a few hundred) can be consumed entirely by reasoning, you’ll get finish_reason: "length" with an empty content and only reasoning_content populated. Either keep max_tokens generous or disable reasoning with reasoning_effort=None for short replies.

Streaming

Set stream: true to receive the response incrementally over server-sent events instead of waiting for the full completion. This is essential for responsive chat UIs and voice-agent pipelines, where you want to start rendering (or speaking) the reply as soon as the first tokens arrive.

Both SDKs return an iterator of chat.completion.chunk objects. Each chunk carries a delta with the new portion of the message, delta.content for the reply text and, when reasoning is enabled, delta.reasoning_content for thinking tokens.

from sarvamai import SarvamAI
client = SarvamAI(
    api_subscription_key="YOUR_SARVAM_API_KEY",
)
stream = client.chat.completions(
    model="sarvam-105b",
    messages=[
        {"role": "user", "content": "Write a short poem about the monsoon."}
    ],
    stream=True,
)
for chunk in stream:
    # The final chunk reports usage and has no choices: guard before indexing
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Over raw HTTP, each event is a data: line containing a chat.completion.chunk JSON object. The final data chunk carries usage (with an empty choices array), and the stream ends with data: [DONE]:

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1699000000,"model":"sarvam-105b","choices":[{"index":0,"delta":{"role":"assistant","content":"The"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1699000000,"model":"sarvam-105b","choices":[{"index":0,"delta":{"content":" rains"},"finish_reason":null}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1699000000,"model":"sarvam-105b","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1699000000,"model":"sarvam-105b","choices":[],"usage":{"prompt_tokens":19,"completion_tokens":3,"total_tokens":22}}
data: [DONE]

When reasoning_effort is set, thinking tokens stream first via delta.reasoning_content, followed by the reply via delta.content. Check both fields if you display reasoning to users.

Tool Calling (Function Calling)

Describe functions your application exposes with the tools parameter, and the model will decide when to call them, returning the function name and JSON arguments instead of (or alongside) a text reply. You execute the function yourself, append the result as a tool message, and call the API again so the model can produce its final answer.

The flow is:

  1. Send the conversation plus tools definitions.
  2. If the model wants a tool, the response has finish_reason: "tool_calls" and message.tool_calls with the function name and stringified JSON arguments.
  3. Run the function, append the assistant message and a {"role": "tool", "tool_call_id": ..., "content": ...} message with the result.
  4. Call the API again, the model answers using the tool output.
import json
from sarvamai import SarvamAI
client = SarvamAI(api_subscription_key="YOUR_SARVAM_API_KEY")
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for an Indian city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name, e.g. Mumbai"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["city"],
            },
        },
    }
]
messages = [{"role": "user", "content": "What's the weather in Mumbai right now?"}]
response = client.chat.completions(
    model="sarvam-105b",
    messages=messages,
    tools=tools,
    tool_choice="auto",
)
message = response.choices[0].message
if message.tool_calls:
    tool_call = message.tool_calls[0]
    args = json.loads(tool_call.function.arguments)
    # Run your actual function here
    weather = {"city": args["city"], "temperature": 31, "condition": "Humid"}
    messages.append(
        {
            "role": "assistant",
            "tool_calls": [
                {
                    "id": tool_call.id,
                    "type": "function",
                    "function": {
                        "name": tool_call.function.name,
                        "arguments": tool_call.function.arguments,
                    },
                }
            ],
        }
    )
    messages.append(
        {
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": json.dumps(weather),
        }
    )
    final = client.chat.completions(
        model="sarvam-105b",
        messages=messages,
        tools=tools,
    )
    print(final.choices[0].message.content)

A tool-call response looks like:

{
  "choices": [
    {
      "index": 0,
      "finish_reason": "tool_calls",
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"city\": \"Mumbai\", \"unit\": \"celsius\"}"
            }
          }
        ]
      }
    }
  ]
}

Controlling tool use with tool_choice

ValueBehavior
"auto" (default when tools are provided)The model decides whether to call a tool or reply directly
"none"The model never calls a tool, tools are ignored
"required"The model must call at least one tool
{"type": "function", "function": {"name": "get_weather"}}Forces the model to call the named function

function.arguments is a JSON string, not an object, always parse it (and validate against your schema) before executing the function.

Structured Outputs (JSON)

The Chat Completions API supports the OpenAI-compatible response_format parameter for getting reliably structured JSON:

response_formatBehavior
{"type": "json_schema", "json_schema": {...}}Structured Outputs: output is constrained to match the JSON Schema you supply (recommended)
{"type": "json_object"}JSON mode: output is guaranteed to be valid JSON, but not a specific schema
{"type": "text"} (default)Plain text output

Structured Outputs with json_schema

Pass a JSON Schema under json_schema.schema, and set "strict": true to enforce adherence. The structured reply arrives as a JSON string in message.content, parse it before use.

import json
from sarvamai import SarvamAI
client = SarvamAI(api_subscription_key="YOUR_SARVAM_API_KEY")
response = client.chat.completions(
    model="sarvam-105b",
    messages=[
        {
            "role": "user",
            "content": "Order: 2 masala dosas and 1 filter coffee to Koramangala, Bengaluru.",
        }
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "food_order",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "items": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "name": {"type": "string"},
                                "quantity": {"type": "integer"},
                            },
                            "required": ["name", "quantity"],
                            "additionalProperties": False,
                        },
                    },
                    "city": {"type": "string"},
                },
                "required": ["items", "city"],
                "additionalProperties": False,
            },
        },
    },
)
order = json.loads(response.choices[0].message.content)
print(order)
# {'items': [{'name': 'masala dosa', 'quantity': 2}, {'name': 'filter coffee', 'quantity': 1}], 'city': 'Bengaluru'}

Both /v1/chat/completions and /v2/chat/completions expose response_format as a typed SDK parameter. On V2, call client.chat.completions_v2; see the Chat Completion V2 API Reference.

The json_schema object accepts:

FieldTypeDescription
namestring (required)Name of the response format. Alphanumeric characters, underscores and dashes only
schemaobjectThe output structure, described as a JSON Schema object
strictbooleanEnable strict schema adherence when generating the output (default false)
descriptionstringWhat the format is for, helps the model decide how to respond

JSON mode with json_object

When you only need valid JSON without enforcing a specific structure, use {"type": "json_object"} and describe the desired shape in your prompt:

curl -X POST https://api.sarvam.ai/v1/chat/completions \
  -H "api-subscription-key: $SARVAM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "sarvam-105b",
    "messages": [
      {"role": "system", "content": "Reply with a JSON object: {\"sentiment\": \"positive\" | \"negative\" | \"neutral\", \"confidence\": number}"},
      {"role": "user", "content": "यह फिल्म शानदार थी!"}
    ],
    "response_format": {"type": "json_object"}
  }'

Even with Structured Outputs, validate the parsed JSON against your expected schema (e.g. with pydantic or zod) before acting on it, the schema constrains the model’s output shape, but your application logic may have stricter requirements (value ranges, business rules, etc.).

Alternative: Tool calling as a JSON schema

If your workflow is already built around tool calling, you can also get structured output by defining a single tool whose parameters schema describes the structure you want, and forcing it with tool_choice. The model’s arguments are then constrained to the schema.

import json
from sarvamai import SarvamAI
client = SarvamAI(api_subscription_key="YOUR_SARVAM_API_KEY")
response = client.chat.completions(
    model="sarvam-105b",
    messages=[
        {
            "role": "user",
            "content": "Order: 2 masala dosas and 1 filter coffee to Koramangala, Bengaluru.",
        }
    ],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "extract_order",
                "description": "Extract a structured food order",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "items": {
                            "type": "array",
                            "items": {
                                "type": "object",
                                "properties": {
                                    "name": {"type": "string"},
                                    "quantity": {"type": "integer"},
                                },
                                "required": ["name", "quantity"],
                            },
                        },
                        "delivery_area": {"type": "string"},
                        "city": {"type": "string"},
                    },
                    "required": ["items", "city"],
                },
            },
        }
    ],
    tool_choice={"type": "function", "function": {"name": "extract_order"}},
)
arguments = response.choices[0].message.tool_calls[0].function.arguments
order = json.loads(arguments)
print(order)
# {'items': [{'name': 'masala dosa', 'quantity': 2}, {'name': 'filter coffee', 'quantity': 1}], 'delivery_area': 'Koramangala', 'city': 'Bengaluru'}

Alternative: Prompt-based JSON

For simple cases, you can also instruct the model to reply with JSON only, set a low temperature, and validate the output before using it (consider JSON mode instead, which guarantees valid JSON):

import json
from sarvamai import SarvamAI
client = SarvamAI(api_subscription_key="YOUR_SARVAM_API_KEY")
response = client.chat.completions(
    model="sarvam-105b",
    messages=[
        {
            "role": "system",
            "content": (
                "Reply with a single JSON object only, no prose, no markdown fences. "
                'Schema: {"sentiment": "positive" | "negative" | "neutral", "confidence": number}'
            ),
        },
        {"role": "user", "content": "यह फिल्म शानदार थी!"},
    ],
    temperature=0.1,
)
raw = response.choices[0].message.content
try:
    result = json.loads(raw)
except json.JSONDecodeError:
    # Retry, or strip markdown fences / extra text before parsing
    raise
print(result)

Always validate model-produced JSON against your expected schema (e.g. with pydantic or zod) and add a retry path, prompt-based JSON is good, but not guaranteed.

API Response Format

Success Response Structure

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1699000000,
  "model": "sarvam-105b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of India is New Delhi. It has been the capital since 1931."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 25,
    "total_tokens": 40
  }
}

Response Fields

FieldTypeDescription
idstringUnique identifier for the completion request
objectstringAlways "chat.completion"
createdintegerUnix timestamp when the completion was created
modelstringThe model used for completion
choices[].indexintegerIndex of the choice in the list
choices[].message.rolestringAlways "assistant"
choices[].message.contentstringThe generated text response (null when the model calls a tool)
choices[].message.reasoning_contentstringThinking steps (only when reasoning_effort is set)
choices[].message.tool_callsarrayTool invocations requested by the model (only when using tool calling)
choices[].finish_reasonstringWhy generation stopped: "stop", "length", "tool_calls", "content_filter", or "function_call" (legacy, only reachable via the deprecated functions parameter, which currently returns 503; use tools/tool_choice instead)
usage.prompt_tokensintegerTokens in the input prompt
usage.completion_tokensintegerTokens in the generated response
usage.total_tokensintegerTotal tokens used (prompt + completion)

Error Responses

All errors return a JSON object with an error field (message, code, request_id). The full error-code table, retry guidance, and SDK exception reference live on the central Errors & Troubleshooting page.

Errors specific to this endpoint:

HTTP StatusError CodeWhen This HappensWhat To Do
400invalid_request_errorMissing messages array or missing model field, invalid parameters, or context overflowInclude both model and a valid messages array; check temperature (0–2) and model name
from sarvamai import SarvamAI
from sarvamai.core.api_error import ApiError
client = SarvamAI(api_subscription_key="YOUR_SARVAM_API_KEY")
try:
    response = client.chat.completions(
        model="sarvam-105b",
        messages=[
            {"role": "user", "content": "What is the capital of India?"}
        ],
    )
    print(response.choices[0].message.content)
except ApiError as e:
    if e.status_code == 400:
        print(f"Bad request: {e.body}")
    elif e.status_code == 403:
        print("Invalid API key. Check your credentials.")
    elif e.status_code == 429:
        print("Rate limit exceeded. Wait and retry.")
    else:
        print(f"Error {e.status_code}: {e.body}")

Limits

LimitValue
Context window128K tokens (sarvam-105b) / 32K tokens (sarvam-105b-conversations)
max_tokensStarter 4096 / Pro 16384 / Business 128000
(reasoning tokens count toward completion tokens)
temperature0–2 (default 0.5 when reasoning is enabled, the default, and 0.2 when reasoning is disabled)
top_p0–1
n (completions per request)1–128
frequency_penalty / presence_penalty-2 to 2
stopUp to 4 sequences
Rate limitsSee Rate Limits

Check out the Chat Completion V1 API Reference to explore Chat Completion and all available options.