Automatically test your agent

​

Overview

You can create customer prompts to guide how customers would interact with your agent and evaluate the results using defined metrics.

​

Steps

​

Step 1: Create a new test case

Click “AI Simulated Chat” to create a new test case.

​

Step 2: Define the customer prompt

Define the customer scenario by providing a prompt. We recommend using this format:

## Identity

Your name is Mike.
Your date of birth is June 10, 1999.
Your order number is 7891273.

## Goal

Your primary objective is to return the package you received and get a refund.

## Personality

You are a patient customer. However, if the conversation becomes too long or complicated, you will show signs of impatience. If the issue remains unresolved, you may become frustrated and angry.

​

Step 3: Run the simulation

Click “Test” to start the conversation.

​

Step 4: Review the results

Manually review the conversation to identify any issues.

​

Step 5: Save as test case

Click “Save” to preserve the test case for future use.

​

Step 6: Define the evaluation metrics

Define the metrics you want to measure. We recommend using this format:

1. Verify that the customer successfully returned the package and received a refund.
2. Confirm that the end_call function was called at the end of the conversation.
3. Ensure the agent's responses are conversational and contain 5 sentences or fewer.

​

Step 7: (Optional) Configure test variables and function mocks

You can specify dynamic variables to be used during testing.

Additionally, you can set up mocks for custom functions to prevent calling actual functions during testing, ensuring consistent results across test runs.