About the Auto QM evaluation criteria

NOTE   This is a new feature that is not yet generally available. If you would like early access to this feature, contact your Provider Account Manager.

Auto QM uses AI to evaluate conversations and provide scores for standard agent performance questions. The scores are automatically generated and available at both individual and aggregated levels. The evaluation for these scores is based on a defined set of criteria.

The conversation is evaluated and scored in the following breakdown of percentages:

• Greeting: 10%

• Empathy: 20%

• Professional language: 20%

• Closing: 10%

• Customer effort: 15%

• Resolution: 25%

NOTE   If an error occurs when processing a question, you must manually add a score for the question or contact an admin for assistance.

When automatically generating a score, Auto QM evaluates the following criteria:

Section	Evaluation questions
Greeting	Did the agent start the conversation with a polite greeting? Did the agent introduce themselves by name, personalizing the conversation and helping to build a connection with the customer?
Empathy	Was empathy required in this conversation? Did the agent express empathy, if required? Was the empathy expressed in a timely and appropriate way by the agent? Did the agent demonstrate a dismissive attitude towards the customer feelings?
Professional language	Did the agent use polite and courteous language throughout the conversation? Did the agent use clear and simple language, avoiding unnecessary jargon, if necessary? Did the agent use any profanity or inappropriate language during the conversation? Did the agent display a negative attitude with a language that contained sarcasm, condescension, or by expressing negative emotions (such as frustration, annoyance, or anger) directly? Was there a talkover event in the conversation that lasted longer than X seconds?
Closing	Did the agent ensure that there are no remaining issues or questions from the customer? Did the agent conclude the conversation with a polite and clear goodbye, ensuring the customer knew the call was ending or being transferred? Did the agent ask for permission and wait for confirmation before transferring the call?
Customer effort	Was the conversation duration over the threshold? Did the agent put the customer on hold for over two minutes at a time? Did the agent put the customer on hold more than X number of times? Was the silence in the conversation from both parties longer than the threshold?
Resolution	Did the agent identify and understand the primary issue? Did the agent offer a solution to address the primary issue of the conversation? Did the customer accept the offered solution?

Calabrio currently presets the following thresholds:

• Individual hold time: > 2 min

• Number of hold events: > 3 holds

• Total silence time: > 10 seconds (exclude silences of less than 2 seconds)

• Conversation duration: > 6 min

The evaluation excludes the following types of conversations:

• Single channel conversations where speakers cannot be separated

• Conversations that are not in English

• Conversations that last less than 30 seconds

• Conversations with fewer than three messages

• Outbound conversations that were spoken on voice mail