Skip to main content
This guide covers the most important design advice and the first things to check when improving a conversation flow agent’s behavior. It is not an exhaustive guide. Before building a conversation flow agent, consider whether you need its added control over task execution or its ability to split instructions and tools across nodes to reduce context. If neither is needed, we recommend starting with a single prompt agent; see When to use single prompt vs. conversation flow agents.

​
Keep it simple

A simple conversation flow is easier to understand, debug, and improve. With fewer branches and instructions to consider, you can more easily trace why the agent behaved a certain way and decide what to change. To keep that complexity manageable, start with the happy path: what the agent needs to accomplish in a typical successful call. Add handling for failures or exceptions only when the need is obvious or supported by evidence. If unsure whether an edge case needs handling, leave it out by default. Keep the prompts and transition conditions simple too. Use short, clear instructions, and add detail only when the task requires it or testing shows it is needed. Avoid prescribing how the agent should respond to every possible variation in what the caller says. The general prompting principles apply to global prompts, node prompts, transition conditions, and tool definitions.

​
Limit transition choices

Simplicity also affects the agent’s accuracy. The more transitions available from a node, the less likely the node transition model is to choose the correct one. Keep the number of choices small. Each global node effectively adds a transition option to every other node in the conversation flow. Adding global nodes therefore increases the number of choices throughout the conversation flow, even at nodes with few transitions of their own. For that reason, use global nodes sparingly for situations that need to be handled regardless of the current node. For example, if a conversation can go poorly at any point, a global catch-all node can handle that situation by transferring the caller to a human or ending the call.

​
Put instructions in the right place

Distinguish between instructions for the entire call, instructions for the current node, and conditions for leaving that node:
  • Global prompt: Describe the agent’s role and instructions that apply throughout the call. Avoid repeating these instructions in individual node prompts.
  • Node prompt: Describe what the agent should say and do in the current node, including when to call its tools. Keep instructions for other nodes out of this prompt, and do not use it to tell the agent to transition.
  • Transition conditions: Describe when the agent should take each transition. Do not include instructions about what to say or which node to go to. Make the conditions clear and avoid overlap that would make multiple transitions apply to the same situation.
This separation matters because transition conditions are evaluated independently of response generation. Conversation flow agents use two models for these tasks. The response model is the model you select for the agent. It generates the agent’s replies. The node transition model decides whether to stay in the current node or transition to another node. Both models see the global prompt, the node prompt, and conversation history. This means the agent can leave a node even if it has not carried out everything in that node’s prompt. The transition conditions determine whether it leaves. If it transitions, the next reply follows the new node’s prompt.

​
Example: Instructions in the wrong place

The Greeting node below shows some common mistakes in how instructions are divided between the node prompt and transition conditions, followed by a corrected version. Before:
Greeting node with transition instructions and booking tasks in its prompt, and destination instructions in its transition conditions

Before: the Greeting prompt includes transitions and tasks for another node.

This version mixes instructions for the Greeting node with directions for transitions and tasks that belong in another node:
  1. The Greeting prompt says “transition to the Booking node,” but the node transition model makes that decision using the configured conditions.
  2. The Greeting prompt also includes date collection and availability instructions. These belong in the Booking node, whose prompt controls the reply after that transition.
  3. Both transition conditions specify a destination. Each transition is already connected to Booking or End Call, so the condition only needs to describe when to take it.
After:
Corrected Greeting node with only the greeting instruction and conditions describing when to transition

After: the prompt describes the greeting, and the conditions describe when to transition.

In the corrected example, the Greeting node’s prompt only instructs the agent to greet the caller and ask how it can help. The transition conditions only describe when to leave the node.

​
Beware of rigid conversation flows

Dividing a task into nodes can help reduce context and control which instructions, tools, and actions are available at each point. However, it can also restrict how the conversation proceeds. Callers may provide several details at once or change an earlier answer, and the connections between nodes may not accommodate that. To allow more flexibility within a task, we recommend using subagent nodes to keep related conversation and tool calls together. A subagent node can handle multiple conversation turns and call tools as needed, so you do not need a separate node for every question or tool call. The deliberately simplified examples below show the restrictions that separate nodes can introduce. The before designs can be appropriate when the task requires that separation. Where it does not, the after designs show how combining nodes allows more flexibility.

​
Example: The caller provides both details at once

Before:
Separate conversation nodes for collecting the caller's name and phone number

Before: separate nodes collect the name and phone number.

This design separates name and phone number collection into two nodes. It requires an extra caller turn when both details arrive together:
  1. The agent is in Collect name and asks for the caller’s name.
  2. The caller gives both their name and phone number in the same reply.
  3. The node transition model transitions to Collect phone number.
  4. It cannot also transition out of Collect phone number from that same reply. The agent will need another caller reply before leaving that node, even though it already has both details.
After:
One subagent node collects both the caller's name and phone number

After: one subagent node collects both contact details.

Combining both questions into one subagent node lets the agent accept both details in the same reply and complete collection without another caller turn.

​
Example: The caller changes their mind

Before:
Separate conversation nodes for choosing pickup or delivery and collecting the details for each option

Before: pickup and delivery are handled in separate nodes with no connection between them.

This design puts pickup and delivery in separate nodes with no transitions between them. That becomes a problem when the caller changes their mind:
  1. The agent asks whether the caller wants to pick up their order.
  2. The caller says yes, and the agent transitions to Collect pickup time.
  3. The caller then says, “Actually, I’d like delivery.”
  4. Collect pickup time has no transition to Collect delivery address, so the agent cannot switch to the node needed for the new request.
After:
One subagent node handles the choice between pickup and delivery and collects the required details

After: one subagent node handles both pickup and delivery.

Combining pickup and delivery into one subagent node lets the agent handle a change of mind within the same node. When the caller switches to delivery, the agent can ask for their delivery address without needing a transition to another node.