Plivo-powered AI voice agents can transform your business operations by enabling natural, intelligent conversations with your customers. By integrating Plivo’s Voice API with Deepgram for speech recognition, OpenAI for conversation processing, and ElevenLabs for natural speech synthesis, you can create sophisticated voice agents that handle customer interactions efficiently.

This integration guide will help you build a voice agent that can:

  • Transcribe live customer audio in real-time
  • Generate contextual responses using AI
  • Convert responses to natural-sounding speech
  • Maintain fluid conversations with built-in voice activity detection

​
Get Started with Plivo

Before developing your AI voice agent, Sign up for Plivo or sign in to your existing account. Purchase a number through the Voice API or Plivo console to receive customer calls.

​
Prerequisites

You’ll need active accounts and API keys from:

​
Clone the Plivo audio stream integration guides repository

git clone https://github.com/plivo/AI-Voice-Agents.git
cd AI-Voice-Agents/Deepgram-openai-elevenlabs

​
Setting Up Your Development Environment

First, create a Python virtual environment and install the required dependencies:

pip install virtualenv
virtualenv -p /usr/bin/python3 venv
source venv/bin/activate

​
Install requirements

  1. Install Python Requirements
pip install -r requirements.txt
  1. Install FFmpeg for audio processing by downloading it from the official website.

​
Configure Your Environment

Create a config.json file with your API credentials:

{
    "auth_id": "your_plivo_auth_id",
    "auth_token": "your_plivo_auth_token",
    "github_token": "your_github_token",
    "deepgram_api_key": "your_deepgram_api_key",
    "openai_api_key": "your_openai_api_key",
    "elevenlabs_api_key": "your_elevenlabs_api_key"
}

​
Customize Your Voice Agent’s Personality

Define your voice agent’s behavior by updating the LLM prompt. You can update the LLM prompt by updating your system message in server.py:

system_msg = """You are John Doe, a voice agent that helps in resolving 
general queries related to any field. When someone says hello, you will greet them and answer their questions in a polite way."""

​
Set Up Local Development Environment

To test your application locally:

  1. Install ngrok:
brew install ngrok  # for macOS
# or download from https://ngrok.com/download for other OS
  1. Start an ngrok tunnel to expose your local server:
ngrok http 5000
  1. Launch your application:
python server.py

​
Configure Your Plivo Number

Set up your Plivo number to handle incoming calls using the command:

python number_setup.py your-plivo-number ws://[your-ngrok-subdomain].ngrok.app/stream

Replace the placeholders with your actual Plivo number and ngrok URL received from step 1.

​
Testing Your Voice Agent

Once configured, your voice agent is ready to handle calls. Place a test call to your Plivo number to interact with your AI-powered voice agent. The agent will:

  1. Listen for incoming audio using voice activity detection
  2. Transcribe speech when a pause is detected
  3. Generate an AI response based on the transcription
  4. Convert the response to natural speech
  5. Play the response back to the caller

This integration enables seamless, natural conversations between your customers and AI voice agents, powered by Plivo’s reliable voice infrastructure.

For additional support: