Prompt Engineering Guide

Prompt Engineering might be the most important part of your agent, it can make or break your agent. This guide will share our findings on how to write prompts that agents can follow more reliably.

This guide is a work in progress and will be updated as we learn more. If you have any suggestions or feedback, please let us know.

To check some of the prompts we have created, you can look at templates in Dashboard (create a new agent and select a template to start).

​

Sectional Prompts

When writing prompts, it is important to break down the system prompts into smaller sections, where each section has its focus, like identity, style, guideline, task & goals. This has a couple of benefits:

• reusable
• easier to maintain
• easier for LLM to understand

## Identify
You are a friendly AI assistant for Retell AI. ....

## Style Guardrails
Be concise: ...
Be conversational: ...
...

## Response Guideline
Return dates in their spoken forms: ...
Ask up to one question at a time: ...
...

## Task
1. Greet the user
...

​

Write Task as Steps

During a call, if you want agent to follow a specific procedure to lead the conversation, we recommend writing the task as steps. This will help LLM understand what to ask at each step and how to proceed. You can have some logic in the steps as well. This also ensures agent does not pack all questions in one go.

## Task
1. Ask for user's name.
2. Ask if user needs a refund, need a replacement, or just retriving information.
  - if user needs a refund, transition to refund state.
  - if user needs a replacement, transition to replacement state.
3. If user is just retriving information, ask for the order number.
...

If you noticed that the agent is still not stopping at the right time, or you need to go through some steps and not stop at each step, you can write it like the following by adding wait for user response to be more explicit:

## Task
1. Inform user why you are calling.
2. Ask for user's name.

wait for user response

3. Ask if user needs a refund, need a replacement, or just retriving information.

wait for user response
  - if user needs a refund, transition to refund state.
  - if user needs a replacement, transition to replacement state.


4. Ask for the order number.

wait for user response
...

​

Break Down Complex Tasks into Multiple States

When the prompt logic gets complicated, and / or a lot of tools are involved, it’s quite easy for LLM to make a wrong decision. Therefore, breaking it down to multiple states is pretty intuitive.

Read more at LLM States. Be sure to write in prompt when to transition to next state, make sure to refer to the state by its name. You can have some duplicate steps in the states so that it’s easier for LLM to locate which step it’s in.

​

Explicitly Write When to Call Tools

A lot of time LLM does not have a clue when to call the tools with a description of what the tool does. It’s important to write in the prompt when to call the tool to ensure an accurate tool calling. Make sure to refer to the tool by its name.

...
2. Ask if user needs a refund, or just retriving information.
  - if user needs a refund, call function transfer_to_support to further assist user.
  - if user needs a replacement, transition to replacement state.
...

​

How To Let LLM Output Nothing

We harded coded a stop sequence in LLM: NO_RESPONSE_NEEDED. Whenever this sequence is met, the response generation would stop. You can then prompt the LLM to output nothing by writing something like:

- if user's name is John, reply exactly the following: "NO_RESPONSE_NEEDED".

This can be useful when you wish for agent to not respond in certain situations (like being put on hold, like call not connected, like dealing with answering machines like Voicemail instructions).