Skip to main content
Resolution criteria decide whether each call in your cohort counts as successful. This is the second step of the Create QA flow, after you define the cohort.
Resolution criteria form answering the question What determines a successful call, with AI evaluated conditions and performance metric sections

The resolution criteria step, where you set what makes a call successful.

​
What determines a successful call?

You define this using two kinds of criteria: AI-evaluated conditions and performance metrics. By default, a call counts as successful only when it meets every condition and every metric. Enable weighted scoring to change how these criteria combine.
1

Add AI-evaluated conditions

An AI-evaluated condition is a qualitative check the AI runs against each call’s transcript and context.
  • Name — A short identifier, such as Call resolved, Customer satisfied, or Issue escalated properly.
  • Prompt description — The prompt the AI uses to judge whether the condition is met, for example AI agent was able to resolve the user's query.
Write prompts that are specific about what success looks like, include context about the call type, and use clear, unambiguous language. Click + Add to add more conditions — each is evaluated independently.
2

Add performance metrics

A performance metric is a quantitative threshold a call must meet. Select a metric and set its threshold:
  • Latency — End-to-end delay between the user speaking and the agent responding.
  • User sentiment — The caller’s emotional state, inferred from speech content, tone, and pitch.
  • Agent sentiment — The emotional tone of the agent’s speech.
  • Overlapping speech — Count of times the user and agent spoke at once.
  • Transcription — Word error rate (WER) and count of mistranscribed entities.
  • Agent hallucination — How often the agent hallucinated.
  • Tool call inaccuracy — Rate at which the agent invoked the wrong tools.
  • Node transition inaccuracy — Rate of incorrect node transitions.
  • Agent naturalness — How human-like the agent sounded across pronunciation, intonation, pacing, and turn-taking.
For full definitions, see AI QA metrics. Click + Add to add more metrics.
Performance metric dropdown with a selected metric and its success threshold value

Example performance metric configuration with a threshold.

Both AI-evaluated conditions and performance metrics are evaluated. Without weighted scoring, a call is successful only if it meets every condition and every metric.
3

Weight your criteria (optional)

By default, every condition and metric is required equally, so a call passes only if it meets all of them. Turn on Weighted scoring to prioritize some criteria over others instead.When enabled, assign a weight to each condition and metric, then set the success threshold the weighted total must reach.
Resolution criteria form with weighted scoring enabled, showing a weight input next to each condition and metric and a success threshold

Weighted scoring with a weight assigned to each condition and metric.

Use weighted scoring when some criteria matter more than others. For example, weight Call resolved higher than Customer satisfied if resolution is your primary goal.
4

Save and run QA

Click Save and Run QA to start analysis. Once it finishes, review the output in View QA results.
If saving fails, check that every required field is filled and every threshold is set, then resolve any validation messages before retrying.