Build a Pipecat voice agent with Sarvam
Want a production voice agent without assembling the stack yourself? Sarvam Voice Agents is a managed platform with speech recognition, reasoning, Indian-language voices, telephony, testing, and analytics included, from ₹3 per minute. This guide shows how to build your own pipeline on the APIs instead.
Overview
This guide demonstrates how to build a real-time voice agent that can listen, understand, and respond naturally using Pipecat for real-time communication and Sarvam AI for speech processing. Perfect for building voice assistants, customer support bots, and conversational AI applications for Indian languages.
Use the Sarvam voice-agents skill to generate correct Pipecat integration code from your AI coding assistant:
See Agent Skills for the full list.
What You’ll Build
A voice agent that can:
- Listen to users speaking (in multiple Indian languages!)
- Understand and process their requests
- Respond back in natural-sounding voices
Quick Overview
- Get API keys (Sarvam)
- Install packages:
pip install "pipecat-ai[daily,sarvam,runner]>=1.14" python-dotenv - Create
.envfile with your API keys - Write ~80 lines of Python code
- Run with appropriate transport
Quick Start
1. Prerequisites
- Python 3.11 or higher (required by
pipecat-ai) - API keys from:
- Sarvam AI (get API key from dashboard)
2. Install Dependencies
macOS/Linux
Windows
3. Create Environment File
Create a file named .env in your project folder and add your API keys:
Replace the values with your actual API keys.
4. Write Your Agent
Create agent.py:
5. Run Your Agent
For Daily transport:
The agent will create a Daily room and provide you with a URL to join.
6. Test Your Agent
Open the provided Daily room URL in your browser and start speaking. Your voice agent will listen and respond!
Customization Examples
Example 1: Hindi Voice Agent
Example 2: Tamil Voice Agent
Example 3: Multilingual Agent (Auto-detect)
Example 4: Speech-to-English Agent (Saaras)
Difference: mode="transcribe" converts speech to text in the same language, while mode="translate" translates speech directly to English text. Use translate mode when the user speaks Indian languages but you want to process/respond in English.
Note: Saaras automatically detects the source language (Hindi, Tamil, etc.) and translates spoken content directly to English text, making Indian language speech comprehensible to English-based LLMs.
Available Options
Language Codes
TTS model and voices
Recommended: model="bulbul:v4-flash" (latest, needs pipecat-ai 1.13.0 or later) with a persona ID in Settings.voice (for example simran_en_customer, ishita_hi_devotional). Browse the Bulbul v4 Flash catalog. If you omit model, Pipecat uses bulbul:v3; if you pick v4 Flash and omit voice, it uses shubh_en_narration_gentle.
Bulbul v3: use model="bulbul:v3" with short names such as shubh, priya, anand — see Voices for the full v3 list.
On current Pipecat versions, language and voice belong in SarvamTTSService.Settings. As bare constructor kwargs they are silently ignored, so the agent falls back to the default voice. See the Pipecat Production Guide for latency tuning.
TTS additional parameters
temperature (0.01–1.0) applies to bulbul:v3 only. bulbul:v4-flash ignores it, so Pipecat doesn’t send it and you can leave it out.
Understanding the Pipeline
Pipecat uses a pipeline architecture where data flows through a series of processors:
- Transport Input: Receives audio from the user
- STT (Speech-to-Text): Converts audio to text using Sarvam’s Saaras
- Context Aggregator (User): Adds user message to conversation context
- LLM: Generates response using Sarvam
- TTS (Text-to-Speech): Converts response to audio using Sarvam’s Bulbul v4 Flash (or v3)
- Transport Output: Sends audio back to the user
- Context Aggregator (Assistant): Saves assistant’s response to context
Pro Tips
- Use
language="unknown"to automatically detect the language. Great for multilingual scenarios! - Sarvam’s models understand code-mixing - your agent can naturally handle Hinglish, Tanglish, and other mixed languages.
- Adjust
paceto customize the voice delivery speed. - Use
sarvam-105bfor the LLM step.
Troubleshooting
API key errors: Check that all keys are in your .env file and the file is in the same directory as your script.
Module not found: Run the installation command again based on your operating system (see Step 2 above).
Poor transcription: Try language="unknown" for auto-detection, or specify the correct language code (en-IN, hi-IN, etc.).
Connection issues: Ensure you have a stable internet connection and the transport is properly configured.
Going to Production?
This guide gets a demo agent talking. Before you ship it, see the Pipecat Production Guide for the defaults you need to change: turn-taking strategies, latency budgets, barge-in tuning, turning off LLM reasoning for voice, reliability, and a full caveats checklist verified against the Sarvam Pipecat plugin source.
Additional Resources
- Sarvam AI Documentation
- Pipecat Documentation
- Pipecat Sarvam LLM Service
- Pipecat GitHub Repository
- Daily.co Documentation
Need Help?
- Sarvam Support: support@sarvam.ai
- Community: Join the Discord Community
Happy Building!