Custom LLM overview

With a custom LLM, you own response generation. Retell handles the call itself (telephony, transcription, turn-taking, speech synthesis) and opens a WebSocket to your server for each call. Retell sends you the live transcript and asks for responses; you stream back what the agent should say.

​

When to use it

Single prompt and conversation flow agents give you built-in tool calling, a knowledge base, warm transfers, and a testing playground with no server to run. They’re the right choice for most agents. Reach for a custom LLM when you need something those frameworks can’t give you:

• Compliance constraints. Prompts and transcripts must be processed inside your own infrastructure, or by a model you host.
• A model Retell doesn’t offer. A fine-tuned model, a self-hosted open-weights model, or a provider outside the supported options.
• Response logic beyond what Retell’s frameworks support. Your own retrieval pipeline instead of Retell’s knowledge base, a multi-step agent loop that runs before the agent speaks, or orchestration that already lives in your own code.

You take on latency, uptime, and reconnection handling. You also give up the features that assume Retell generates the responses:

• The LLM playground can’t test a custom LLM agent, and simulation and batch testing reject them. Web and phone calls are the only way to test.
• Agent transfer can’t target a custom LLM agent.
• Your own tool calls only appear in the transcript if you report them yourself. Retell records the actions it runs (end_call, transfer_call, press_digit) automatically.

​

How a call flows

One WebSocket is opened per call, to llm_websocket_url with the call ID appended as the last path segment. Retell drives the conversation and tells you when it needs something. Retell decides when the agent speaks, so not every response you generate gets used. If the caller keeps talking, Retell discards the response it asked for and asks again with a new response_id.

​

What your server implements

Retell tags every message it sends with interaction_type: You tag everything you send with response_type: The LLM WebSocket reference has the full field-by-field spec for every event above.

​

Example servers

Runnable reference implementations, both with function calling:

• Node.js (Express) — OpenAI, Azure OpenAI, OpenRouter
• Python (FastAPI) — OpenAI, Claude

​

Next steps