Skip to navigation
API Guides & TutorialsLarge Language ModelsHow-to

How to control the response length with max_tokens

View as Markdown

The max_tokens parameter lets you control how long the model’s response can be: in terms of tokens.

  • A token can be a word, part of a word, or even punctuation
    (Example: “Hello!” = 2 tokens: "Hello" + "!")

Why use max_tokens?

  • To limit the size of the output
  • To control latency / cost (fewer tokens = faster and cheaper)
  • To avoid overly long answers if you want concise responses

How to choose the value:

Parameter details:

ParameterTypeDefault
max_tokensInteger2048

First, install the SDK:

pip install -Uqq sarvamai

Then use the following Python code:

from sarvamai import SarvamAI
# Initialize the SarvamAI client with your API key
client = SarvamAI(api_subscription_key="YOUR_SARVAM_API_KEY")
# Example 1: Using default max_tokens (not specified): defaults to 2048
response = client.chat.completions(
    model="sarvam-105b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Tell me about the planet Mars."}
    ],
    # max_tokens not specified → defaults to 2048
)
print(response.choices[0].message.content)
from sarvamai import SarvamAI
client = SarvamAI(api_subscription_key="YOUR_SARVAM_API_KEY")
# Example 2: Using max_tokens = 100: limit response length
response = client.chat.completions(
    model="sarvam-105b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Summarize the plot of Mahabharata."}
    ],
    max_tokens=100
)
# Receive assistant's reply as output
print(response.choices[0].message.content)