Tests API overview | Sarvam API Docs

A test suite is a set of test cases for an agent. When you run a suite, the platform plays a simulated user against a committed version of your agent, one conversation per test case, and an AI judge reads each finished transcript and grades it against the behaviors you expect. This API does everything the dashboard’s Tests tab does — build suites, run them, and read the results.

For the dashboard walkthrough of the same concepts, see Tests under Build.

What makes up a test suite

Part	What it is
Test case	One simulated conversation to grade: a user scenario plus the expected behaviors
User scenario	A brief for the simulated user — who they are, what they want, and how they behave
Expected behaviors	The checks the judge grades for that test case, each `Pass` or `Fail`
Category	A label that groups related test cases
Global guardrails	One-line rules graded on every test case in the suite, such as “never gives medical advice”
Variables	Common values that can be reused across test cases
Variable overrides	A test case’s own value for a suite variable, used for that case only
Variable mapping	Connects suite variables to the agent’s own variables for one app version
Max turns	The longest a simulated conversation can run before it is cut off
Run	One execution of the suite (or one test case) against a committed app version
Frequency	How many times each test case executes within a run

Base URL

https://apps.sarvam.ai/api/evals/v1/{org_id}/{workspace_id}/test-suites

Suites are workspace-scoped. A suite created without an app_id is reusable across every agent in the workspace; a suite created with one is bound to that app, and each app can have at most one bound suite.

Requests are authenticated with your API key, as everywhere else in the Voice Agents API — see Introduction.

Rate limits

Limit	Scope	Applies to
60 requests / minute	Per API key, within an org and workspace	Every Tests endpoint
10 runs / 5 minutes	Per organization	Run a test suite
20 runs / minute	Per organization	Run a single test case

The per-organization run limits are shared across every key and every user in the org. Exceeding any limit returns 429 with a Retry-After header giving the window in seconds:

HTTP/1.1 429 Too Many Requests
Retry-After: 60

Honour Retry-After rather than retrying immediately. If your workload needs more headroom, higher per-key limits can be provisioned per organization — contact Sarvam support.

How the pieces fit together

Create a suite

Create a test suite with a name, shared variables, global_guardrails, and optionally up to 200 test cases inline.

Add and refine test cases

Add a test case with a user_scenario and expected_behaviors; update or duplicate cases as the suite evolves.

Map suite variables onto the app

Create a variable mapping so the suite’s variables reach the agent’s own variables. Without a mapping, runs execute with the agent’s defaults.

Run

Run a test suite against an app_id and committed app_version. The call returns 202 with a run_id immediately; execution happens in the background.

Poll

Get a run until status is completed or failed. completed_executions against total_executions is your progress bar. The response also shows, for each test case, how many of its executions have passed so far.

Read the results

Get test case results returns one entry per execution, each with the full conversation transcript and a verdict for every expected behavior.

Errors

Most errors carry a detail field with a message string:

{ "detail": "Test suite already exists for app_id: clinic-front-desk" }

The exception is 402 — returned by the run endpoints when the workspace has no remaining credits — which carries details (plural) instead:

{ "details": "Insufficient credits. Please top up your wallet to continue." }

Status	When
`400`	The app did not answer the pre-run ping — check the `app_id` and `app_version`
`401`	Missing or invalid `X-API-Key`
`402`	No remaining credits; top up to resume runs
`403`	The key is not valid for this org or workspace
`404`	No such suite, test case, run, or mapping in this workspace
`409`	Creating an app-bound suite for an app that already has one
`429`	Rate limit exceeded — honour `Retry-After`

What this API does not do

These are deliberate, not gaps to work around:

No deletion. Suites and test cases are created and edited through the API but removed only in the dashboard.
No run cancellation. A queued run executes to completion; poll it rather than trying to stop it.
No AI suite generation. Generating test cases from a description is a dashboard feature.
No export. Read results as JSON from the results endpoint.

Best practices

Writing scenarios and behaviors that grade reliably, and getting variables to reach the agent.

Tests in the dashboard

The same suites, runs, and results in the Canvas.