Skip to main content
resolution_criteria.png

​
What Determines a Successful Call?

The “What determines a successful call?” question is the core of defining resolution criteria. This step allows you to set up the evaluation framework that determines whether calls in your cohort are considered successful or not.
1

AI Evaluated Condition

The AI Evaluated Condition section allows you to define custom criteria that are evaluated by AI based on call transcripts and context.NameThe Name field is a short identifier for your condition (e.g., “Call resolved”, “Customer satisfied”, “Issue escalated properly”).Prompt DescriptionThe Prompt Description is the actual prompt that the AI uses to evaluate whether the condition is met for each call.
  • Example: “AI agent was able to resolve user’s query”
  • Best Practice:
    • Be specific about what success looks like
    • Include relevant context about the call type or use case
    • Use clear, unambiguous language
You can add multiple AI Evaluated Conditions by clicking the ”+ Add” button. Each condition will be evaluated independently, and you can delete any condition using the trash icon.
2

Performance Metric

The Performance Metric section allows you to define quantitative thresholds that calls must meet to be considered successful.

​
Metrics

The Metric dropdown lets you select which performance metric to evaluate:
  • Latency: Measures the end-to-end delay between a user speaking and the Voice AI beginning its spoken response.
  • User Sentiment: Represents the emotional state of the caller as inferred from speech content, tone, and pitch.
  • Agent Sentiment: Represents the emotional tone expressed by the Voice AI during speech output.
  • Interruptions: Count of times the user interrupted the agent.
  • Transcription: WER and number of mistranscribed entities.
  • Agent Hallucination: Measures how often the agent hallucinated.
  • Tool Call Inaccuracy: Measures the rate at which the agent invoked incorrect tools.
  • Node Transition Inaccuracy: Measures incorrect node transitions.
  • Agent Naturalness: Measures how human-like the agent sounded, including pronunciation, intonation, pacing, turn-taking behavior, and the absence of robotic patterns.

​
Example Configuration

performance-metric.png
You can add multiple Performance Metrics by clicking the ”+ Add” button.
Both AI Evaluated Conditions and Performance Metrics are evaluated. A call is considered successful only if it meets all defined criteria across both sections.
3

Weighted Scoring (Advanced)

The Weighted Scoring toggle enables you to assign different weights to your resolution criteria, allowing you to give more weight to certain conditions or metrics over others.
  • When Enabled: You can assign weights to each AI Evaluated Condition and Performance Metric. Set the Success Criteria threshold.
  • When Disabled: All criteria are treated equally - a call must meet all conditions to be considered successful
resolution-criteria-weighted.png
Use weighted scoring when some criteria are more important than others. For example, you might weight “Call resolved” higher than “Customer satisfaction” if resolution is your primary goal.
4

Save and Run QA

After configuring your resolution criteria, complete the process by clicking the Save and Run QA button.
If you encounter an error when saving, review your conditions and metrics to ensure all required fields are filled and criteria thresholds are set. Resolve any validation messages before retrying.