Build a Retell agent: configuration overview

Building an agent means deciding how it understands callers, decides what to say, sounds when it speaks, and handles the parts of a call beyond talking, like voicemail and transfers. This page orients you across all of it. Start with choosing an agent type, since that decision shapes which other settings apply.

​

Choose an agent type

Every agent is either a voice agent (talks on phone calls) or a chat agent (text-only, reachable via API, an embeddable widget, or SMS). For your first voice agent, we recommend starting with a single prompt. See Choose an agent type for when to use conversation flow instead.

​

Understand what callers say

Retell picks a speech recognition provider for your agent’s configured languages. You can also tune how transcription behaves mid-call and filter out background noise so it doesn’t get transcribed as speech.

​

Generate better responses from the LLM

Start with the general prompting principles to keep instructions clear and concise. You can also use Agent Handbook presets for specific behaviors and Conversational Mode for more natural replies. Enable these when the agent needs them. LLM options control model-level settings, including temperature and fast tier.

​

Customize the voice

Pick a platform or custom voice, including cloned voices, and tune how it delivers speech: expressiveness, speech controls like pauses and spelling, pausing silently when the caller asks it to wait, and a fallback TTS provider if the primary one fails.

​

Handle telephony scenarios

Configure what the agent does when a call isn’t a normal back-and-forth: detect and respond to voicemail and IVR menus, capture DTMF keypad input from the caller, and pass metadata between systems with custom SIP headers.

​

Give your agent context

Attach a knowledge base of URLs, documents, or text so the agent retrieves accurate answers instead of relying only on the prompt, and inject per-call data like names or account IDs with dynamic variables.

​

Add guardrails

Turn on guardrails to automatically catch prohibited topics in the agent’s responses and jailbreak attempts from the caller, replacing them with a safe message instead of ending or transferring the call.

​

Test and deploy

Once the agent is configured, validate it in the LLM Playground or simulation testing, then purchase a number or connect your own via SIP to take calls. If you manage agent configuration in code, follow the TypeScript SDK agent creation guide to create the response engine and agent, test the draft, and publish a version.