> For clean Markdown of any page, append `.md` to the page URL.
> For a complete documentation index, see https://docs.sarvam.ai/llms.txt.
> For full documentation content in one file, see https://docs.sarvam.ai/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.sarvam.ai/_mcp/server.

# Tests API overview

A **test suite** is a set of test cases for an agent. When you run a suite, the platform plays a simulated user against a committed version of your agent, one conversation per test case, and an AI judge reads each finished transcript and grades it against the behaviors you expect. This API does everything the dashboard's Tests tab does — build suites, run them, and read the results.

For the dashboard walkthrough of the same concepts, see [Tests](/conversations/build/tests) under Build.

## What makes up a test suite

| Part                   | What it is                                                                                  |
| ---------------------- | ------------------------------------------------------------------------------------------- |
| **Test case**          | One simulated conversation to grade: a user scenario plus the expected behaviors            |
| **User scenario**      | A brief for the simulated user — who they are, what they want, and how they behave          |
| **Expected behaviors** | The checks the judge grades for that test case, each `Pass` or `Fail`                       |
| **Category**           | A label that groups related test cases                                                      |
| **Global guardrails**  | One-line rules graded on every test case in the suite, such as "never gives medical advice" |
| **Variables**          | Common values that can be reused across test cases                                          |
| **Variable overrides** | A test case's own value for a suite variable, used for that case only                       |
| **Variable mapping**   | Connects suite variables to the agent's own variables for one app version                   |
| **Max turns**          | The longest a simulated conversation can run before it is cut off                           |
| **Run**                | One execution of the suite (or one test case) against a committed app version               |
| **Frequency**          | How many times each test case executes within a run                                         |

## Base URL

```
https://apps.sarvam.ai/api/evals/v1/{org_id}/{workspace_id}/test-suites
```

Suites are **workspace-scoped**. A suite created without an `app_id` is reusable across every agent in the workspace; a suite created with one is bound to that app, and each app can have at most one bound suite.

Requests are authenticated with your API key, as everywhere else in the Voice Agents API — see [Introduction](/conversations/api/introduction).

## Rate limits

| Limit                    | Scope                                    | Applies to                                                            |
| ------------------------ | ---------------------------------------- | --------------------------------------------------------------------- |
| **60 requests / minute** | Per API key, within an org and workspace | Every Tests endpoint                                                  |
| **10 runs / 5 minutes**  | Per organization                         | [Run a test suite](/conversations/api/tests/runs/run-suite)           |
| **20 runs / minute**     | Per organization                         | [Run a single test case](/conversations/api/tests/runs/run-test-case) |

The per-organization run limits are shared across every key and every user in the org. Exceeding any limit returns `429` with a `Retry-After` header giving the window in seconds:

```
HTTP/1.1 429 Too Many Requests
Retry-After: 60
```

Honour `Retry-After` rather than retrying immediately. If your workload needs more headroom, higher per-key limits can be provisioned per organization — contact Sarvam support.

## How the pieces fit together

#### Create a suite

[Create a test suite](/conversations/api/tests/create) with a name, shared `variables`, `global_guardrails`, and optionally up to 200 test cases inline.

#### Add and refine test cases

[Add a test case](/conversations/api/tests/test-cases/add) with a `user_scenario` and `expected_behaviors`; [update](/conversations/api/tests/test-cases/update) or [duplicate](/conversations/api/tests/test-cases/duplicate) cases as the suite evolves.

#### Map suite variables onto the app

[Create a variable mapping](/conversations/api/tests/mappings/create) so the suite's variables reach the agent's own variables. Without a mapping, runs execute with the agent's defaults.

#### Run

[Run a test suite](/conversations/api/tests/runs/run-suite) against an `app_id` and committed `app_version`. The call returns `202` with a `run_id` immediately; execution happens in the background.

#### Poll

[Get a run](/conversations/api/tests/runs/get) until `status` is `completed` or `failed`. `completed_executions` against `total_executions` is your progress bar. The response also shows, for each test case, how many of its executions have passed so far.

#### Read the results

[Get test case results](/conversations/api/tests/runs/results) returns one entry per execution, each with the full conversation transcript and a verdict for every expected behavior.

## Errors

Most errors carry a `detail` field with a message string:

```json
{ "detail": "Test suite already exists for app_id: clinic-front-desk" }
```

The exception is `402` — returned by the run endpoints when the workspace has no remaining credits — which carries `details` (plural) instead:

```json
{ "details": "Insufficient credits. Please top up your wallet to continue." }
```

| Status | When                                                                           |
| ------ | ------------------------------------------------------------------------------ |
| `400`  | The app did not answer the pre-run ping — check the `app_id` and `app_version` |
| `401`  | Missing or invalid `X-API-Key`                                                 |
| `402`  | No remaining credits; top up to resume runs                                    |
| `403`  | The key is not valid for this org or workspace                                 |
| `404`  | No such suite, test case, run, or mapping in this workspace                    |
| `409`  | Creating an app-bound suite for an app that already has one                    |
| `429`  | Rate limit exceeded — honour `Retry-After`                                     |

## What this API does not do

These are deliberate, not gaps to work around:

* **No deletion.** Suites and test cases are created and edited through the API but removed only in the dashboard.
* **No run cancellation.** A queued run executes to completion; poll it rather than trying to stop it.
* **No AI suite generation.** Generating test cases from a description is a dashboard feature.
* **No export.** Read results as JSON from the results endpoint.

## Next

#### [Best practices](/conversations/api/tests/best-practices)

Writing scenarios and behaviors that grade reliably, and getting variables to reach the agent.

#### [Tests in the dashboard](/conversations/build/tests)

The same suites, runs, and results in the Canvas.