> For clean Markdown of any page, append `.md` to the page URL.
> For a complete documentation index, see https://docs.sarvam.ai/llms.txt.
> For full documentation content in one file, see https://docs.sarvam.ai/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.sarvam.ai/_mcp/server.

# Build with Sarvam AI in LangChain

> Use the official langchain-sarvam package to call Sarvam chat models from LangChain: install, setup, and code examples for invoke, streaming, tool calling, structured output, and reasoning mode.

## Overview

**[LangChain](https://python.langchain.com)** is a Python framework for building LLM applications, giving you a single set of interfaces (`invoke`, `stream`, `bind_tools`, `with_structured_output`, and more) that work the same way across different model providers. Instead of learning each provider's own SDK, you write your app logic once against LangChain's `BaseChatModel` interface and swap providers by changing the model you instantiate.

**[langchain-sarvam](https://github.com/sarvamai/langchain-sarvam)** is the official LangChain integration package for Sarvam AI. It wraps the `sarvamai` Python SDK's **chat completion** API behind LangChain's `ChatSarvam` class, so Sarvam's `sarvam-105b` and `sarvam-30b` models work as a drop-in `BaseChatModel` that you can use in chains, agents, retrievers, tool-calling loops, and structured-output pipelines exactly like `ChatOpenAI` or `ChatAnthropic` (including LangSmith tracing support).

## Quick start

### 1. Install

```bash
pip install langchain-sarvam python-dotenv
```

Requires Python 3.10+. `python-dotenv` loads `SARVAM_API_KEY` from a local `.env` file, used throughout the examples below.

### 2. Set your API key

Get an API key from **[dashboard.sarvam.ai](https://dashboard.sarvam.ai)** and add it to your environment:

```bash title=".env"
SARVAM_API_KEY="your_api_key"
```

### 3. Create a chat model instance

```python
import os

from dotenv import load_dotenv
from langchain_sarvam import ChatSarvam

load_dotenv()

if not os.getenv("SARVAM_API_KEY"):
    raise RuntimeError("Set SARVAM_API_KEY in your environment or .env file before running.")

llm = ChatSarvam(model="sarvam-105b")
```

`ChatSarvam` reads `SARVAM_API_KEY` from the environment automatically, so no explicit client setup is needed. Pass `api_key="..."` directly if you'd rather not use an environment variable.

### 4. Invoke

```python
import os

from dotenv import load_dotenv
from langchain_sarvam import ChatSarvam

load_dotenv()

if not os.getenv("SARVAM_API_KEY"):
    raise RuntimeError("Set SARVAM_API_KEY in your environment or .env file before running.")

llm = ChatSarvam(model="sarvam-30b")

try:
    response = llm.invoke("Translate this to malayalam: 'Keep cooking, guys'")
    print(response.content)  # പാചകം തുടരൂ, സുഹൃത്തുക്കളേ
except Exception as e:
    print(f"Error: {e}")
```

If longer completions come back truncated mid-sentence, the model is hitting its output token limit. Raise `max_tokens` and check `finish_reason` to confirm when a response was truncated:

```python
import os

from dotenv import load_dotenv
from langchain_sarvam import ChatSarvam

load_dotenv()

if not os.getenv("SARVAM_API_KEY"):
    raise RuntimeError("Set SARVAM_API_KEY in your environment or .env file before running.")

llm = ChatSarvam(model="sarvam-30b", max_tokens=4096)

try:
    response = llm.invoke("Write a detailed explanation of...")
    if response.response_metadata.get("finish_reason") == "length":
        print("Response was truncated; consider raising max_tokens further.")
except Exception as e:
    print(f"Error: {e}")
```

***

## Streaming

```python
import os

from dotenv import load_dotenv
from langchain_sarvam import ChatSarvam

load_dotenv()

if not os.getenv("SARVAM_API_KEY"):
    raise RuntimeError("Set SARVAM_API_KEY in your environment or .env file before running.")

llm = ChatSarvam(model="sarvam-105b", streaming=True)

try:
    for chunk in llm.stream("Count from 1 to 5."):
        print(chunk.content, end="", flush=True)
except Exception as e:
    print(f"Error: {e}")
```

Token usage is emitted on the final chunk, so `chunk.usage_metadata` is populated once streaming completes, no extra configuration needed.

## Async

```python
import asyncio
import os

from dotenv import load_dotenv
from langchain_sarvam import ChatSarvam

load_dotenv()

if not os.getenv("SARVAM_API_KEY"):
    raise RuntimeError("Set SARVAM_API_KEY in your environment or .env file before running.")

llm = ChatSarvam(model="sarvam-105b")


async def main():
    try:
        response = await llm.ainvoke("What is the capital of India?")
        print(response.content)

        async for chunk in llm.astream("Count from 1 to 3."):
            print(chunk.content, end="", flush=True)
    except Exception as e:
        print(f"Error: {e}")


asyncio.run(main())
```

## Tool calling

`ChatSarvam` supports LangChain's standard `bind_tools()` interface:

```python
import os

from dotenv import load_dotenv
from langchain_core.tools import tool
from langchain_sarvam import ChatSarvam

load_dotenv()

if not os.getenv("SARVAM_API_KEY"):
    raise RuntimeError("Set SARVAM_API_KEY in your environment or .env file before running.")


@tool
def get_weather(city: str) -> str:
    """Get the weather for a city."""
    return f"Sunny in {city}"


llm = ChatSarvam(model="sarvam-105b")
llm_with_tools = llm.bind_tools([get_weather])

try:
    response = llm_with_tools.invoke("What is the weather in Mumbai?")
    print(response.tool_calls)
except Exception as e:
    print(f"Error: {e}")
```

Pass `tool_choice="auto"` (model decides), `tool_choice="none"` (disabled), `tool_choice="required"` (must call a tool), or a tool name string to force a specific tool.

## Structured output

```python
import os

from dotenv import load_dotenv
from pydantic import BaseModel
from langchain_sarvam import ChatSarvam

load_dotenv()

if not os.getenv("SARVAM_API_KEY"):
    raise RuntimeError("Set SARVAM_API_KEY in your environment or .env file before running.")


class CapitalCity(BaseModel):
    city: str
    country: str


llm = ChatSarvam(model="sarvam-105b")
structured_llm = llm.with_structured_output(CapitalCity)

try:
    result = structured_llm.invoke("What is the capital of France?")
    print(result)  # CapitalCity(city='Paris', country='France')
except Exception as e:
    print(f"Error: {e}")
```

`with_structured_output()` sends an explicit `reasoning_effort=None` by default, unless the model or the call already sets `reasoning_effort`. Sarvam's reasoning models default to `reasoning_effort="medium"` server-side, and the resulting `reasoning_content` competes with the structured JSON payload for the same completion token budget. Disabling reasoning by default keeps that budget free for the structured output. Pass `reasoning_effort="low"`, `"medium"`, or `"high"` explicitly (on the model or the call) to combine reasoning with structured output.

## Reasoning mode

Pass `reasoning_effort` to control how much effort the model puts into reasoning before responding: `"low"`, `"medium"`, or `"high"`. Leaving it unset lets the API apply its own default.

```python
import os

from dotenv import load_dotenv
from langchain_sarvam import ChatSarvam

load_dotenv()

if not os.getenv("SARVAM_API_KEY"):
    raise RuntimeError("Set SARVAM_API_KEY in your environment or .env file before running.")

reasoning_llm = ChatSarvam(model="sarvam-105b", reasoning_effort="high")

try:
    response = reasoning_llm.invoke("Explain the Riemann hypothesis.")
    print(response.additional_kwargs.get("reasoning_content"))
    print(response.content)
except Exception as e:
    print(f"Error: {e}")
```

See [Reasoning effort](/api/api-guides-tutorials/chat-completion/how-to/adjust-the-models-thinking-level) for details on the allowed values.

***

## Capabilities at a glance

| `ChatSarvam` interface   | LangChain concept                 | Underlying Sarvam capability                                                                          |
| ------------------------ | --------------------------------- | ----------------------------------------------------------------------------------------------------- |
| `invoke` / `ainvoke`     | Sync/async single-turn completion | [Chat completion](/api/api-guides-tutorials/chat-completion/overview)                                 |
| `stream` / `astream`     | Sync/async token streaming        | Chat completion (streaming)                                                                           |
| `bind_tools`             | Tool/function calling             | Chat completion (tool calling)                                                                        |
| `with_structured_output` | Schema-constrained output         | Chat completion ([Structured Outputs](/api-reference/chat/chat-completions) via `response_format`)    |
| `reasoning_effort` param | Extended reasoning                | [Reasoning effort](/api/api-guides-tutorials/chat-completion/how-to/adjust-the-models-thinking-level) |

Supported models: [Sarvam 105B](/api/getting-started/models/sarvam-105b) (flagship, 128K context) and [Sarvam 30B](/api/getting-started/models/sarvam-30b) (64K context).

***

## Troubleshooting

#### ValueError: Sarvam API key not found

Set `SARVAM_API_KEY` in the environment the process actually reads (`.env` loaded, no stray whitespace in the key), or pass `api_key="..."` explicitly when constructing `ChatSarvam`.

#### 401/403 from Sarvam

Sarvam's API returns `403` (not `401`) for auth failures; see [Errors & Troubleshooting](/api/getting-started/errors-troubleshooting).

#### Calling a non-default environment

Set `SARVAM_API_BASE_URL`, or pass `base_url="..."` to `ChatSarvam`, to point at a different API base URL than `https://api.sarvam.ai/v1`.

#### Responses get cut off partway through

Output length is capped by `max_tokens` (unset by default, so the API's own default applies). Raise it and check `response.response_metadata["finish_reason"] == "length"` to confirm truncation; see the [Invoke](#4-invoke) section above.

#### ValueError: Got unknown message type

`ChatSarvam` only converts `SystemMessage`, `HumanMessage`, `AIMessage`, and `ToolMessage`. This surfaces when older-style `FunctionMessage`s or a generic `ChatMessage` end up in the history, commonly from a prebuilt agent or migrated codebase, since those aren't converted. Map them to `ToolMessage`/`HumanMessage` before passing the history to `ChatSarvam`.

#### Catching a specific error type (e.g. rate limits) never triggers

`ChatSarvam` wraps every exception from the underlying `sarvamai` client, including auth, rate limit, and network errors, in a single `ChatSarvamError`. Catch `ChatSarvamError` rather than a provider-specific exception type, and inspect `str(e)` for the underlying cause.

***

## Additional resources

* [langchain-sarvam on GitHub](https://github.com/sarvamai/langchain-sarvam) (source, issues, changelog)
* [langchain-sarvam on PyPI](https://pypi.org/project/langchain-sarvam/)
* [LangChain chat models documentation](https://python.langchain.com/docs/concepts/chat_models/)
* [Chat completion overview](/api/api-guides-tutorials/chat-completion/overview)
* [Build with Sarvam AI in the Vercel AI SDK](/api/integration/vercel-ai-sdk) (same chat capability from a TypeScript app)
* [Build workflows with Sarvam AI in n8n](/api/integration/n8n) (same Sarvam capabilities without writing code)

***

## Need help?

* Sarvam Support: [developer@sarvam.ai](mailto:developer@sarvam.ai)
* Community: [Join the Discord Community](https://discord.com/invite/5rAsykttcs)