Collection Agent using LiveKit

View as Markdown

Overview

This guide demonstrates how to build a voice-based collection agent that can handle payment reminders, follow-ups, and payment assistance using LiveKit for real-time communication and Sarvam AI for speech processing. Perfect for fintech companies, banks, and lending institutions serving Indian customers.

For the broader architecture pattern this agent fits into, telephony ingress, escalation, latency targets, see the IVR & Contact Center use-case guide.

What You’ll Build

A collection agent that can:

  • Make professional payment reminder calls in multiple Indian languages
  • Handle customer queries about payments, due dates, and payment options
  • Guide customers through payment processes
  • Maintain a professional and empathetic tone

Quick Overview

  1. Get API keys (LiveKit, Sarvam)
  2. Install packages
  3. Create .env file with your API keys
  4. Write the agent code
  5. Run: python agent.py dev
  6. Test: python agent.py console

Quick Start

1. Prerequisites

2. Install Dependencies

pip install "livekit-agents[sarvam,silero]" python-dotenv

3. Create Environment File

Create a file named .env in your project folder and add your API keys:

LIVEKIT_URL=wss://your-project-xxxxx.livekit.cloud
LIVEKIT_API_KEY=APIxxxxxxxxxxxxx
LIVEKIT_API_SECRET=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
SARVAM_API_KEY=sk_xxxxxxxxxxxxxxxxxxxxxxxx

Replace the values with your actual API keys.

4. Write Your Agent

Create collection_agent.py:

import logging
from dotenv import load_dotenv
from livekit.agents import JobContext, WorkerOptions, cli
from livekit.agents.voice import Agent, AgentSession
from livekit.plugins import sarvam
# Load environment variables
load_dotenv()
# Set up logging
logger = logging.getLogger("collection-agent")
logger.setLevel(logging.INFO)
class CollectionAgent(Agent):
def __init__(self) -> None:
super().__init__(
# Collection agent personality and instructions
instructions="""
You are a professional and empathetic collection agent working for ABC Bank.
Customer Account Details:
- Bank Name: ABC Bank
- EMI Amount: ₹5,000
- Due Date: 15th January 2025
- Loan Type: Personal Loan
- Account Status: Payment Overdue
Your responsibilities:
- Remind customers about their pending EMI payment of ₹5,000 which was due on 15th January
- Provide information about payment due dates, amounts, and available payment methods
- Help customers understand their payment options and any applicable late fees
- Guide customers through the payment process if they want to pay immediately
- Address customer concerns about their account with empathy
- Offer payment plans or extensions when appropriate (mention that you can connect them with a specialist)
Payment Methods to mention:
- UPI payment to ABC Bank
- Net Banking
- ABC Bank mobile app
- Visit nearest ABC Bank branch
Communication guidelines:
- Always maintain a professional yet friendly tone
- Be empathetic to customer's financial situations
- Never be aggressive, threatening, or use inappropriate language
- If a customer is upset, remain calm and understanding
- Speak clearly and concisely
- Confirm important details like EMI amount (₹5,000) and due date (15th January)
- If customer requests to speak to a human, acknowledge and offer to transfer
Start by greeting the customer, introducing yourself as calling from ABC Bank,
and politely remind them about their pending EMI of ₹5,000.
""",
# Saaras v4 STT - Converts speech to text
stt=sarvam.STT(
language="unknown", # Auto-detect language
model="saaras:v4",
mode="transcribe"
),
# Sarvam LLM - The "brain" that processes and generates responses
llm=sarvam.LLM(model="sarvam-105b"),
# Bulbul TTS - Converts text to speech
tts=sarvam.TTS(
language_code="en-IN",
model="bulbul:v3",
speaker="aditya" # Professional male voice
),
)
async def on_enter(self):
"""Called when user joins - agent starts the conversation"""
self.session.generate_reply()
async def entrypoint(ctx: JobContext):
"""Main entry point - LiveKit calls this when a user connects"""
logger.info(f"User connected to room: {ctx.room.name}")
# Create and start the agent session
session = AgentSession()
await session.start(
agent=CollectionAgent(),
room=ctx.room
)
if __name__ == "__main__":
# Run the agent
cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

5. Run Your Agent

python collection_agent.py dev

6. Test Your Agent

In a new terminal, run:

python collection_agent.py console

Customization Examples

Example 1: Hindi Collection Agent

For customers who prefer Hindi:

stt=sarvam.STT(
language="hi-IN", # Hindi
model="saaras:v4",
mode="transcribe"
),
tts=sarvam.TTS(
language_code="hi-IN",
model="bulbul:v3",
speaker="anand" # Professional Hindi male voice
)

Example 2: Tamil Collection Agent

stt=sarvam.STT(language="ta-IN", model="saaras:v4", mode="transcribe"),
tts=sarvam.TTS(
language_code="ta-IN",
model="bulbul:v3",
speaker="priya"
)

Example 3: Multilingual Agent (Auto-detect)

stt=sarvam.STT(language="unknown", model="saaras:v4", mode="transcribe"), # Auto-detects language
tts=sarvam.TTS(language_code="en-IN", model="bulbul:v3", speaker="aditya")

Available Options

Language Codes

LanguageCode
English (India)en-IN
Hindihi-IN
Bengalibn-IN
Tamilta-IN
Telugute-IN
Gujaratigu-IN
Kannadakn-IN
Malayalamml-IN
Marathimr-IN
Punjabipa-IN
Odiaod-IN
Auto-detectunknown

Speaker Voices (Bulbul v3)

Male (23): Shubh (default), Aditya, Rahul, Rohan, Amit, Dev, Ratan, Varun, Manan, Sumit, Kabir, Aayan, Ashutosh, Advait, Anand, Tarun, Sunny, Mani, Gokul, Vijay, Mohit, Rehan, Soham

Female (14): Ritu, Priya, Neha, Pooja, Simran, Kavya, Ishita, Shreya, Roopa, Tanya, Shruti, Suhani, Kavitha, Rupali


Pro Tips

  • Use language="unknown" to automatically detect the language - great for diverse customer bases
  • Use a professional male voice like aditya or anand for collection calls
  • Sarvam’s models handle code-mixing naturally - customers can switch between languages mid-conversation
  • Always maintain compliance with collection regulations in your jurisdiction

Troubleshooting

API key errors: Check that all keys are in your .env file and the file is in the same directory as your script.

Module not found: Run the installation command again based on your operating system.

Poor transcription: Try language="unknown" for auto-detection, or specify the correct language code.


Additional Resources


Need Help?


Happy Building!