> For clean Markdown of any page, append `.md` to the page URL.
> For a complete documentation index, see https://docs.sarvam.ai/llms.txt.
> For full documentation content in one file, see https://docs.sarvam.ai/llms-full.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.sarvam.ai/_mcp/server.

# Run a test suite

POST https://apps.sarvam.ai/api/evals/v1/{org_id}/{workspace_id}/test-suites/{test_suite_id}/runs
Content-Type: application/json

Queues every test case in the suite and returns immediately with a `run_id`. Poll `GET /test-suites/{test_suite_id}/runs/{run_id}` until `status` is `completed` or `failed`, then read a test case's results for its transcript and verdicts.

Reference: https://docs.sarvam.ai/conversations/api/tests/runs/run-suite

## Request

### Path parameters

- `org_id` (string, required) — Organization ID
- `workspace_id` (string, required) — Workspace ID
- `test_suite_id` (string, required)

### Headers

- `X-API-Key` (string, required) — Your workspace API key.

### Body (application/json)

This endpoint expects a RunCreate.

- `app_id` (string, required) — App to run against.
- `app_version` (integer, required) — Version of the app to run against.
- `frequency` (integer, optional, default: 1) — How many times to execute each test case.
- `run_name` (string, optional, nullable)
- `on_end_eval` (boolean, optional, default: true) — Grade expected behaviors once the conversation ends.

## Response

### 202

Successful Response

- `run_id` (string, required) — Poll `GET /test-suites/{test_suite_id}/runs/{run_id}` with this.
- `test_suite_id` (string, required)
- `scope` (enum, required)
  - Allowed values: `suite`, `test_case`
- `status` (string, required) — `created`, `running`, `completed` or `failed`.

## Errors

### 400 Bad Request Error

Unable to reach the app.

- `error` (ErrorBody, required) — Details of the error that occurred.

### 401 Unauthorized Error

No Bearer token or X-API-Key was supplied, or it is invalid.

- `error` (ErrorBody, required) — Details of the error that occurred.

### 402 Payment Required Error

The workspace has no remaining credits. Top up to resume evaluations.

- `details` (string, required) — Human-readable description of the failure.

### 403 Forbidden Error

The credential is not valid for this org or workspace.

- `error` (ErrorBody, required) — Details of the error that occurred.

### 404 Not Found Error

No such resource in this org and workspace.

- `error` (ErrorBody, required) — Details of the error that occurred.

### 422 Unprocessable Entity Error

Validation Error

- `detail` (list of ValidationError, optional)

### 429 Too Many Requests Error

Rate limit exceeded for this caller.

- `error` (ErrorBody, required) — Details of the error that occurred.

## Types

### ErrorBody

Details of the error that occurred.

- `code` (string, required) — A service-specific error code.
- `message` (string, required) — A human-readable description of the error.

### ValidationError

- `loc` (list of ValidationErrorLocItems, required)
- `msg` (string, required)
- `type` (string, required)

### ValidationErrorLocItems

## Examples

**Request**

```json
{
  "app_id": "clinic-front-desk",
  "app_version": 4,
  "frequency": 2,
  "run_name": "Pre-release check, v4",
  "on_end_eval": true
}
```

**Response**

```json
{
  "run_id": "01K5T9C4XR7NB2QJ5WD8HKTM0E",
  "test_suite_id": "01K5T8A2MZQ4VE6XCJ0RDBHW7N",
  "scope": "suite",
  "status": "running"
}
```

**SDK Code**

```python Run queued
import requests

url = "https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites/test_suite_id/runs"

payload = {
    "app_id": "clinic-front-desk",
    "app_version": 4,
    "frequency": 2,
    "run_name": "Pre-release check, v4",
    "on_end_eval": True
}
headers = {
    "X-API-Key": "<your-api-key>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.json())
```

```javascript Run queued
const url = 'https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites/test_suite_id/runs';
const options = {
  method: 'POST',
  headers: {'X-API-Key': '<your-api-key>', 'Content-Type': 'application/json'},
  body: '{"app_id":"clinic-front-desk","app_version":4,"frequency":2,"run_name":"Pre-release check, v4","on_end_eval":true}'
};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go Run queued
package main

import (
	"fmt"
	"strings"
	"net/http"
	"io"
)

func main() {

	url := "https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites/test_suite_id/runs"

	payload := strings.NewReader("{\n  \"app_id\": \"clinic-front-desk\",\n  \"app_version\": 4,\n  \"frequency\": 2,\n  \"run_name\": \"Pre-release check, v4\",\n  \"on_end_eval\": true\n}")

	req, _ := http.NewRequest("POST", url, payload)

	req.Header.Add("X-API-Key", "<your-api-key>")
	req.Header.Add("Content-Type", "application/json")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby Run queued
require 'uri'
require 'net/http'

url = URI("https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites/test_suite_id/runs")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Post.new(url)
request["X-API-Key"] = '<your-api-key>'
request["Content-Type"] = 'application/json'
request.body = "{\n  \"app_id\": \"clinic-front-desk\",\n  \"app_version\": 4,\n  \"frequency\": 2,\n  \"run_name\": \"Pre-release check, v4\",\n  \"on_end_eval\": true\n}"

response = http.request(request)
puts response.read_body
```

```java Run queued
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.post("https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites/test_suite_id/runs")
  .header("X-API-Key", "<your-api-key>")
  .header("Content-Type", "application/json")
  .body("{\n  \"app_id\": \"clinic-front-desk\",\n  \"app_version\": 4,\n  \"frequency\": 2,\n  \"run_name\": \"Pre-release check, v4\",\n  \"on_end_eval\": true\n}")
  .asString();
```

```php Run queued
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('POST', 'https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites/test_suite_id/runs', [
  'body' => '{
  "app_id": "clinic-front-desk",
  "app_version": 4,
  "frequency": 2,
  "run_name": "Pre-release check, v4",
  "on_end_eval": true
}',
  'headers' => [
    'Content-Type' => 'application/json',
    'X-API-Key' => '<your-api-key>',
  ],
]);

echo $response->getBody();
```

```csharp Run queued
using RestSharp;

var client = new RestClient("https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites/test_suite_id/runs");
var request = new RestRequest(Method.POST);
request.AddHeader("X-API-Key", "<your-api-key>");
request.AddHeader("Content-Type", "application/json");
request.AddParameter("application/json", "{\n  \"app_id\": \"clinic-front-desk\",\n  \"app_version\": 4,\n  \"frequency\": 2,\n  \"run_name\": \"Pre-release check, v4\",\n  \"on_end_eval\": true\n}", ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
```

```swift Run queued
import Foundation

let headers = [
  "X-API-Key": "<your-api-key>",
  "Content-Type": "application/json"
]
let parameters = [
  "app_id": "clinic-front-desk",
  "app_version": 4,
  "frequency": 2,
  "run_name": "Pre-release check, v4",
  "on_end_eval": true
] as [String : Any]

let postData = JSONSerialization.data(withJSONObject: parameters, options: [])

let request = NSMutableURLRequest(url: NSURL(string: "https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites/test_suite_id/runs")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "POST"
request.allHTTPHeaderFields = headers
request.httpBody = postData as Data

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```