Skip to main content

How does a voice AI agent work?

A voice agent turns a phone call or web session into a conversation by connecting four layers:
  1. Telephony. The call arrives over the phone network, over a SIP trunk you already own, or starts in a browser or app.
  2. Speech to text. The caller’s audio is transcribed while they talk.
  3. Language model. The transcript goes to an LLM that decides what to say next, using the agent’s prompt, tools, and knowledge base.
  4. Text to speech. The reply is turned back into audio and streamed to the caller with minimal delay.
Wiring those four together yourself is the easy part. Keeping the conversation natural is not: the agent has to know when the caller has finished speaking, recover when it is interrupted, hold up through background noise, and fall back cleanly when a provider degrades mid-call. Retell runs all four layers and handles that orchestration for you, so you build an agent as a single prompt or a conversation flow instead of assembling providers. See orchestration overview for how the layers fit together, and speech recognition, LLM options, and voices for the choices at each one.

Key capabilities

Lead qualification. Screen inbound leads on the phone, ask the qualifying questions, and book or route only the ones that fit. Start with a single prompt agent. Customer support. Answer common questions, look up order or account details in your systems, and hand the complex cases to a person. Start with a single prompt agent. AI receptionist and appointment scheduling. Answer after hours, check availability, and book, reschedule, or cancel appointments. Start with a single prompt agent. Patient scheduling and intake. Handle the structure healthcare calls need, including intake questions and confirmations. Consider a conversation flow agent if compliance requires a fixed order of steps. Dispatch and field service. Take service requests, capture the job details, and route them to the right technician or crew. Start with a single prompt agent. Agencies and resellers. Build agents for multiple client accounts from one platform, each with its own numbers, prompts, and analytics. Either agent type.

Build

Retell agents use a single prompt or a conversation flow, with call handling, a knowledge base, and integrations built in. We recommend starting with a single prompt; see Choose an agent type for when conversation flow is useful. The build overview covers the rest of the configuration.
single-prompt

Single Prompt Agent

Define your whole agent with one prompt. Best for most agents, and the recommended starting point.
conversation-flow

Conversation Flow Agent

Divide the task into nodes for tighter execution control or less context per node. Best when you need a fixed, auditable order of steps, like compliance-driven calls.
multi-modality

Multi-modality

Voice agents for calls, chat agents for text.
call-center-features

Call center features

Warm transfer, IVR navigation, DTMF capture, and more.
knowledge-base

Knowledge base

Crawl a website or upload documents your agent retrieves from.
integrations

Out-of-the-box integrations

CRM, helpdesk, calendar, and file-storage integrations: connect once, available to every agent.
post-call-analysis

Post Call Extraction

Built-in call success and sentiment scoring, plus custom fields synced to your CRM.
conductor

Conductor

Retell’s AI copilot that builds, tests, and refines your agent with you.

Test

Test in text or real audio, manually or automatically, before an agent ever takes a real call. See the testing overview to compare every method.
test-call

Testing tools

Chat, call, or debug your agent in the Playground.
simulation-testing

Simulation testing

Run graded test cases to catch regressions before launch.
ab-testing

A/B testing

Compare agents or prompts by splitting live traffic between them.

Deploy

purchase-phone-number

Purchase phone number

Buy a Retell-managed US or Canada number, no provider account needed.
sip-trunking

Bring your own telephony

Connect your own provider over SIP trunking.
voice-widget

Voice widget

Embed a voice call widget on your website.
create-web-call

Web call

Connect a web call from your frontend with the Web SDK.

Monitor

live-call-monitoring

Live call monitoring

Follow live transcripts, listen in, or take over a call.
custom-analytics

Custom analytics

Build custom dashboards for your call and chat metrics.
ai-qa-overview

AI QA

Automatically score production calls for quality and accuracy.

Data

call-history

Call & chat history

Browse past calls and chats with transcripts and analysis.
crm

CRM

Sync contacts and push call data to Salesforce, HubSpot, Dynamics 365, GoHighLevel, or Zoho CRM automatically.

Developer friendly

Build entirely from the dashboard with no code, or go fully programmatic with the API, SDKs, and an MCP server.

API reference

Manage agents, calls, and numbers programmatically.

SDKs

Official Node.js and Python client libraries.

MCP server

Manage agents from Cursor, Claude Desktop, and more.

Webhook

Real-time call events, no polling required.

Next up

Ready to dive in? Start here.

Quickstart

Build your first phone agent in 15 minutes.

Voice Orchestration

How STT, LLM, and TTS orchestrate into one voice stack.

FAQ

Retell handles turn-taking, interruptions, and background noise as part of orchestration, which is what separates a natural call from a bot that talks over people. You choose the voice from the platform voices, and expressive mode adds natural pacing and emotion.
Yes. Retell is HIPAA compliant, and a signed BAA is required before you transmit protected health information. Retell is also SOC 2 Type 1 and Type 2 certified and GDPR compliant. See compliance for data handling and the erasure process.
Hand it to a person. Retell supports cold, warm, and agentic warm call transfer, so the agent can route the caller immediately or brief the person taking over first.
Yes. Connect your own provider over SIP with custom telephony and keep the numbers you have, or buy a number from Retell if you would rather not manage a provider.