> For clean Markdown of any page, append `.md` to the page URL. > For a complete documentation index, see https://docs.sarvam.ai/llms.txt. > For full documentation content in one file, see https://docs.sarvam.ai/llms-full.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.sarvam.ai/_mcp/server. # Create a test suite POST https://apps.sarvam.ai/api/evals/v1/{org_id}/{workspace_id}/test-suites Content-Type: application/json Set `app_id` to bind the suite to one app; omit it for a suite reusable across apps. Reference: https://docs.sarvam.ai/api-reference/tests/create ## Request ### Path parameters - `org_id` (string, required) — Organization ID - `workspace_id` (string, required) — Workspace ID ### Headers - `X-API-Key` (string, required) — Your workspace API key. ### Body (application/json) This endpoint expects a TestSuiteCreate. - `name` (string, required) - `description` (string, optional, nullable) - `app_id` (string, optional, nullable) — Bind the suite to one app. Omit for a workspace-level suite reusable across apps. - `variables` (list of SuiteVariable, optional, default: []) - `global_guardrails` (list of string, optional, default: []) — Behaviors every test case in the suite is graded against. At most 20, each up to 500 characters. - `test_cases` (list of TestCaseSpec, optional, default: []) — At most 200. Add more later through the test-case endpoint. ## Response ### 201 Successful Response - `test_suite_id` (string, required) ## Errors ### 401 Unauthorized Error No Bearer token or X-API-Key was supplied, or it is invalid. - `error` (ErrorBody, required) — Details of the error that occurred. ### 403 Forbidden Error The credential is not valid for this org or workspace. - `error` (ErrorBody, required) — Details of the error that occurred. ### 409 Conflict Error An app-level suite already exists for this app. - `error` (ErrorBody, required) — Details of the error that occurred. ### 422 Unprocessable Entity Error Validation Error - `detail` (list of ValidationError, optional) ### 429 Too Many Requests Error Rate limit exceeded for this caller. - `error` (ErrorBody, required) — Details of the error that occurred. ## Types ### SuiteVariable - `name` (string, required) - `value` (string, required) ### TestCaseSpec A test case supplied inline when creating a suite. - `name` (string, required) - `category` (string, required) - `user_scenario` (string, required) — What the simulated user is trying to do. - `expected_behaviors` (list of Behavior, optional, default: []) — Behaviors the agent is graded against. - `variable_overrides` (list of SuiteVariable, optional, default: []) — Suite variables overridden for this test case only. - `max_turns` (integer, optional, default: 50) ### ErrorBody Details of the error that occurred. - `code` (string, required) — A service-specific error code. - `message` (string, required) — A human-readable description of the error. ### ValidationError - `loc` (list of ValidationErrorLocItems, required) - `msg` (string, required) - `type` (string, required) ### Behavior - `name` (string, required) - `description` (string, required) ### ValidationErrorLocItems ## Examples **Request** ```json { "name": "Front desk — booking flows", "description": "Rescheduling, cancellations, and identity checks for the clinic agent.", "app_id": "clinic-front-desk", "variables": [ { "name": "clinic_name", "value": "Sunrise Dental" }, { "name": "patient_name", "value": "Ravi" } ], "global_guardrails": [ "Never gives medical advice; clinical questions are directed to a doctor.", "Confirms the patient's full name and phone number before changing a booking." ], "test_cases": [ { "name": "Caller reschedules to the weekend", "category": "Rescheduling", "user_scenario": "The caller has an appointment tomorrow morning but now wants a weekend slot. They are in a hurry, switch to Hindi midway, and ask twice whether a cancellation fee applies.", "expected_behaviors": [ { "name": "Offers concrete slots", "description": "Proposes at least two specific dates and times instead of asking the caller to choose blindly." }, { "name": "Confirms the new booking", "description": "Repeats the final date, time and clinic name back to the caller before ending the call." } ], "variable_overrides": [ { "name": "patient_name", "value": "Meera" } ], "max_turns": 30 } ] } ``` **Response** ```json { "test_suite_id": "01K5T8A2MZQ4VE6XCJ0RDBHW7N" } ``` **SDK Code** ```python Suite created import requests url = "https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites" payload = { "name": "Front desk — booking flows", "description": "Rescheduling, cancellations, and identity checks for the clinic agent.", "app_id": "clinic-front-desk", "variables": [ { "name": "clinic_name", "value": "Sunrise Dental" }, { "name": "patient_name", "value": "Ravi" } ], "global_guardrails": ["Never gives medical advice; clinical questions are directed to a doctor.", "Confirms the patient's full name and phone number before changing a booking."], "test_cases": [ { "name": "Caller reschedules to the weekend", "category": "Rescheduling", "user_scenario": "The caller has an appointment tomorrow morning but now wants a weekend slot. They are in a hurry, switch to Hindi midway, and ask twice whether a cancellation fee applies.", "expected_behaviors": [ { "name": "Offers concrete slots", "description": "Proposes at least two specific dates and times instead of asking the caller to choose blindly." }, { "name": "Confirms the new booking", "description": "Repeats the final date, time and clinic name back to the caller before ending the call." } ], "variable_overrides": [ { "name": "patient_name", "value": "Meera" } ], "max_turns": 30 } ] } headers = { "X-API-Key": "", "Content-Type": "application/json" } response = requests.post(url, json=payload, headers=headers) print(response.json()) ``` ```javascript Suite created const url = 'https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites'; const options = { method: 'POST', headers: {'X-API-Key': '', 'Content-Type': 'application/json'}, body: '{"name":"Front desk — booking flows","description":"Rescheduling, cancellations, and identity checks for the clinic agent.","app_id":"clinic-front-desk","variables":[{"name":"clinic_name","value":"Sunrise Dental"},{"name":"patient_name","value":"Ravi"}],"global_guardrails":["Never gives medical advice; clinical questions are directed to a doctor.","Confirms the patient\'s full name and phone number before changing a booking."],"test_cases":[{"name":"Caller reschedules to the weekend","category":"Rescheduling","user_scenario":"The caller has an appointment tomorrow morning but now wants a weekend slot. They are in a hurry, switch to Hindi midway, and ask twice whether a cancellation fee applies.","expected_behaviors":[{"name":"Offers concrete slots","description":"Proposes at least two specific dates and times instead of asking the caller to choose blindly."},{"name":"Confirms the new booking","description":"Repeats the final date, time and clinic name back to the caller before ending the call."}],"variable_overrides":[{"name":"patient_name","value":"Meera"}],"max_turns":30}]}' }; try { const response = await fetch(url, options); const data = await response.json(); console.log(data); } catch (error) { console.error(error); } ``` ```go Suite created package main import ( "fmt" "strings" "net/http" "io" ) func main() { url := "https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites" payload := strings.NewReader("{\n \"name\": \"Front desk — booking flows\",\n \"description\": \"Rescheduling, cancellations, and identity checks for the clinic agent.\",\n \"app_id\": \"clinic-front-desk\",\n \"variables\": [\n {\n \"name\": \"clinic_name\",\n \"value\": \"Sunrise Dental\"\n },\n {\n \"name\": \"patient_name\",\n \"value\": \"Ravi\"\n }\n ],\n \"global_guardrails\": [\n \"Never gives medical advice; clinical questions are directed to a doctor.\",\n \"Confirms the patient's full name and phone number before changing a booking.\"\n ],\n \"test_cases\": [\n {\n \"name\": \"Caller reschedules to the weekend\",\n \"category\": \"Rescheduling\",\n \"user_scenario\": \"The caller has an appointment tomorrow morning but now wants a weekend slot. They are in a hurry, switch to Hindi midway, and ask twice whether a cancellation fee applies.\",\n \"expected_behaviors\": [\n {\n \"name\": \"Offers concrete slots\",\n \"description\": \"Proposes at least two specific dates and times instead of asking the caller to choose blindly.\"\n },\n {\n \"name\": \"Confirms the new booking\",\n \"description\": \"Repeats the final date, time and clinic name back to the caller before ending the call.\"\n }\n ],\n \"variable_overrides\": [\n {\n \"name\": \"patient_name\",\n \"value\": \"Meera\"\n }\n ],\n \"max_turns\": 30\n }\n ]\n}") req, _ := http.NewRequest("POST", url, payload) req.Header.Add("X-API-Key", "") req.Header.Add("Content-Type", "application/json") res, _ := http.DefaultClient.Do(req) defer res.Body.Close() body, _ := io.ReadAll(res.Body) fmt.Println(res) fmt.Println(string(body)) } ``` ```ruby Suite created require 'uri' require 'net/http' url = URI("https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites") http = Net::HTTP.new(url.host, url.port) http.use_ssl = true request = Net::HTTP::Post.new(url) request["X-API-Key"] = '' request["Content-Type"] = 'application/json' request.body = "{\n \"name\": \"Front desk — booking flows\",\n \"description\": \"Rescheduling, cancellations, and identity checks for the clinic agent.\",\n \"app_id\": \"clinic-front-desk\",\n \"variables\": [\n {\n \"name\": \"clinic_name\",\n \"value\": \"Sunrise Dental\"\n },\n {\n \"name\": \"patient_name\",\n \"value\": \"Ravi\"\n }\n ],\n \"global_guardrails\": [\n \"Never gives medical advice; clinical questions are directed to a doctor.\",\n \"Confirms the patient's full name and phone number before changing a booking.\"\n ],\n \"test_cases\": [\n {\n \"name\": \"Caller reschedules to the weekend\",\n \"category\": \"Rescheduling\",\n \"user_scenario\": \"The caller has an appointment tomorrow morning but now wants a weekend slot. They are in a hurry, switch to Hindi midway, and ask twice whether a cancellation fee applies.\",\n \"expected_behaviors\": [\n {\n \"name\": \"Offers concrete slots\",\n \"description\": \"Proposes at least two specific dates and times instead of asking the caller to choose blindly.\"\n },\n {\n \"name\": \"Confirms the new booking\",\n \"description\": \"Repeats the final date, time and clinic name back to the caller before ending the call.\"\n }\n ],\n \"variable_overrides\": [\n {\n \"name\": \"patient_name\",\n \"value\": \"Meera\"\n }\n ],\n \"max_turns\": 30\n }\n ]\n}" response = http.request(request) puts response.read_body ``` ```java Suite created import com.mashape.unirest.http.HttpResponse; import com.mashape.unirest.http.Unirest; HttpResponse response = Unirest.post("https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites") .header("X-API-Key", "") .header("Content-Type", "application/json") .body("{\n \"name\": \"Front desk — booking flows\",\n \"description\": \"Rescheduling, cancellations, and identity checks for the clinic agent.\",\n \"app_id\": \"clinic-front-desk\",\n \"variables\": [\n {\n \"name\": \"clinic_name\",\n \"value\": \"Sunrise Dental\"\n },\n {\n \"name\": \"patient_name\",\n \"value\": \"Ravi\"\n }\n ],\n \"global_guardrails\": [\n \"Never gives medical advice; clinical questions are directed to a doctor.\",\n \"Confirms the patient's full name and phone number before changing a booking.\"\n ],\n \"test_cases\": [\n {\n \"name\": \"Caller reschedules to the weekend\",\n \"category\": \"Rescheduling\",\n \"user_scenario\": \"The caller has an appointment tomorrow morning but now wants a weekend slot. They are in a hurry, switch to Hindi midway, and ask twice whether a cancellation fee applies.\",\n \"expected_behaviors\": [\n {\n \"name\": \"Offers concrete slots\",\n \"description\": \"Proposes at least two specific dates and times instead of asking the caller to choose blindly.\"\n },\n {\n \"name\": \"Confirms the new booking\",\n \"description\": \"Repeats the final date, time and clinic name back to the caller before ending the call.\"\n }\n ],\n \"variable_overrides\": [\n {\n \"name\": \"patient_name\",\n \"value\": \"Meera\"\n }\n ],\n \"max_turns\": 30\n }\n ]\n}") .asString(); ``` ```php Suite created request('POST', 'https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites', [ 'body' => '{ "name": "Front desk — booking flows", "description": "Rescheduling, cancellations, and identity checks for the clinic agent.", "app_id": "clinic-front-desk", "variables": [ { "name": "clinic_name", "value": "Sunrise Dental" }, { "name": "patient_name", "value": "Ravi" } ], "global_guardrails": [ "Never gives medical advice; clinical questions are directed to a doctor.", "Confirms the patient\'s full name and phone number before changing a booking." ], "test_cases": [ { "name": "Caller reschedules to the weekend", "category": "Rescheduling", "user_scenario": "The caller has an appointment tomorrow morning but now wants a weekend slot. They are in a hurry, switch to Hindi midway, and ask twice whether a cancellation fee applies.", "expected_behaviors": [ { "name": "Offers concrete slots", "description": "Proposes at least two specific dates and times instead of asking the caller to choose blindly." }, { "name": "Confirms the new booking", "description": "Repeats the final date, time and clinic name back to the caller before ending the call." } ], "variable_overrides": [ { "name": "patient_name", "value": "Meera" } ], "max_turns": 30 } ] }', 'headers' => [ 'Content-Type' => 'application/json', 'X-API-Key' => '', ], ]); echo $response->getBody(); ``` ```csharp Suite created using RestSharp; var client = new RestClient("https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites"); var request = new RestRequest(Method.POST); request.AddHeader("X-API-Key", ""); request.AddHeader("Content-Type", "application/json"); request.AddParameter("application/json", "{\n \"name\": \"Front desk — booking flows\",\n \"description\": \"Rescheduling, cancellations, and identity checks for the clinic agent.\",\n \"app_id\": \"clinic-front-desk\",\n \"variables\": [\n {\n \"name\": \"clinic_name\",\n \"value\": \"Sunrise Dental\"\n },\n {\n \"name\": \"patient_name\",\n \"value\": \"Ravi\"\n }\n ],\n \"global_guardrails\": [\n \"Never gives medical advice; clinical questions are directed to a doctor.\",\n \"Confirms the patient's full name and phone number before changing a booking.\"\n ],\n \"test_cases\": [\n {\n \"name\": \"Caller reschedules to the weekend\",\n \"category\": \"Rescheduling\",\n \"user_scenario\": \"The caller has an appointment tomorrow morning but now wants a weekend slot. They are in a hurry, switch to Hindi midway, and ask twice whether a cancellation fee applies.\",\n \"expected_behaviors\": [\n {\n \"name\": \"Offers concrete slots\",\n \"description\": \"Proposes at least two specific dates and times instead of asking the caller to choose blindly.\"\n },\n {\n \"name\": \"Confirms the new booking\",\n \"description\": \"Repeats the final date, time and clinic name back to the caller before ending the call.\"\n }\n ],\n \"variable_overrides\": [\n {\n \"name\": \"patient_name\",\n \"value\": \"Meera\"\n }\n ],\n \"max_turns\": 30\n }\n ]\n}", ParameterType.RequestBody); IRestResponse response = client.Execute(request); ``` ```swift Suite created import Foundation let headers = [ "X-API-Key": "", "Content-Type": "application/json" ] let parameters = [ "name": "Front desk — booking flows", "description": "Rescheduling, cancellations, and identity checks for the clinic agent.", "app_id": "clinic-front-desk", "variables": [ [ "name": "clinic_name", "value": "Sunrise Dental" ], [ "name": "patient_name", "value": "Ravi" ] ], "global_guardrails": ["Never gives medical advice; clinical questions are directed to a doctor.", "Confirms the patient's full name and phone number before changing a booking."], "test_cases": [ [ "name": "Caller reschedules to the weekend", "category": "Rescheduling", "user_scenario": "The caller has an appointment tomorrow morning but now wants a weekend slot. They are in a hurry, switch to Hindi midway, and ask twice whether a cancellation fee applies.", "expected_behaviors": [ [ "name": "Offers concrete slots", "description": "Proposes at least two specific dates and times instead of asking the caller to choose blindly." ], [ "name": "Confirms the new booking", "description": "Repeats the final date, time and clinic name back to the caller before ending the call." ] ], "variable_overrides": [ [ "name": "patient_name", "value": "Meera" ] ], "max_turns": 30 ] ] ] as [String : Any] let postData = JSONSerialization.data(withJSONObject: parameters, options: []) let request = NSMutableURLRequest(url: NSURL(string: "https://apps.sarvam.ai/api/evals/v1/org_id/workspace_id/test-suites")! as URL, cachePolicy: .useProtocolCachePolicy, timeoutInterval: 10.0) request.httpMethod = "POST" request.allHTTPHeaderFields = headers request.httpBody = postData as Data let session = URLSession.shared let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in if (error != nil) { print(error as Any) } else { let httpResponse = response as? HTTPURLResponse print(httpResponse) } }) dataTask.resume() ```