Skip to main content
A batch test run is a set of test cases run together in a simulation, so you can check many scenarios in one pass instead of one at a time. Batches are how you use your saved test cases as a regression suite: run them after every prompt or flow change, and confirm the pass rate holds before you deploy. For example, a support team keeps two dozen test cases covering refunds, escalations, and identity checks. Before each release they run the whole set as a batch, then compare the pass rate to the previous run to catch anything a prompt change broke.

​
Run a batch

In the Test Cases tab, select the cases you want to run (one, several, or all) and click Run Test. Retell launches a batch, even for a single case, and sends the results to Batch Testing History.

Selecting test cases and running them as a batch.

​
Track a running batch

You track and review batches under the Batch Testing History tab. A batch that’s still running is marked Ongoing in the list on the left, and a Running tests indicator sits below its results table until the last case finishes. Results stream in on a 10-second refresh rather than all at once: each case joins the results table as it completes, and the counts above the table go up with it. Cases that haven’t finished yet stay hidden, so the table only ever shows completed runs.

​
Review results

Runs land in Batch Testing History. The ledger on the left lists every run, each showing:
  • the number of test cases in the run
  • the date and time it finished
  • the pass rate (the share of cases that met all their success criteria)
The list only ever shows batches run against one agent version at a time, and the version chip above it tells you which. It defaults to your latest version, so older batches won’t be listed until you switch. To see another version’s history, select that version in the agent’s version panel; to go back to the latest, clear the chip with its X. Select a run to open its results table, and filter it by All, Passed, or Failed. An Error count appears alongside them when a run failed to complete instead of being graded, such as a simulation that timed out. Its columns are: Select a test case row to open its details panel on the right, which shows:
  • the test case ID and test result
  • one explanation of why the run passed or failed across all of its success criteria
  • the full run transcript: node transitions, tool calls and their responses, and the agent and user turns
A View In Test Playground button opens the agent in a new tab with that run’s transcript loaded into the Playground, where you can edit any turn and continue the conversation by hand.

​
Manage batches with the API

Run and read batches programmatically to fold them into CI before you deploy: The simulation testing page walks through this as a full CI sequence.

​
Next steps