Behavioral Signal Processing for Contact Centers

Over a decade ago, SAIL became the first research group to demonstrate automatic recognition of emotion from audio data collected in call centers⁶, conducting research that was eventually awarded with the ICMI Ten Year Technical Impact Award in 2014. Using speech recordings from a call center database consisting of over 1000 calls and a over 7000 utterances, the researchers employed signal processing and machine learning to recognize negative emotions . This novel work demonstrated that it isn’t enough to look at the words that customers or agents were speaking, but also how the person sounds . Measuring the intonation and volume of the speakers more than halved the emotion recognition error (from 25% to 11%)! We at Behavioral Signals have spent over a decade improving the reliability of emotion recognition systems from text, speech, and even body gesture and facial expression. These affecting computing tools form the basis of behavioral signal processing techniques and are applied to a wide-range of use-cases and verticals.

Recruiting, training and retaining top agent talent is the biggest headache in modern contact centers. As the complexity of product offerings and agent-customer increases being able to select and train agents that understand both what to say and how to communicate their message gets increasingly hard. Being emotionally intelligent, empathetic, sociable are some of the top qualities for an agent that takes hours to train and distill. Now with behavioral signal processing it is possible to identify and measure these qualities in every call, as well as select good and bad examples of these (and many more) behaviours in order to educate agent using learning by example. Researcher at SAIL have been able to accurately measure empathy in counseling using speech analytics and create an application for rating the therapist⁷. Using these same technologies Behavioral Signals revolutionizes call monitoring and the quality assurance process in contact centers today. In addition to measuring the overall quality of the call and the matrix of competencies that an agent displays during calls, behavioral signal processing can also provide metrics for compliance, fraud detection and agent wellness.

In a recent article, the Harvard Business Review states that “an emotional connection matters more than customer satisfaction⁸” for business growth. Specifically “ emotional motivators provide a better gauge of customer future value to a firm than any other metric including brand awareness and customer satisfaction⁹.” Emotion and behavioral metrics can provide unique insights on the future value of the customer, as well as accurately predict customer satisfaction, churn/loyalty related metrics and -most importantly- propensity to buy/pay. How probable is that a customer will give your product a positive referral or a negative review in the next week? Will that customer increase or decrease his volume of transactions with your financial institution over the next month? Will a customer pay as promised on time or default on a loan? These and many other questions can be answered with greater accuracy than ever before using behavioral signals processing. Contact centers can turn this information into actionable intelligence to tune their campaigns scripts, adapt their calling strategies (dialers), and improve their product and loyalty offerings.

Obtaining a snapshot of the quality of agent-customer interaction for each campaign, product line and agent group is essential for monitoring the health of your call-center and taking the appropriate executive decisions. For example in Fig. 3 we show a snapshot of Behavioral Signals call emotion recognition software platform (callER) that provides a snapshot of the agent & customer behaviors, as well as key indicators (KPIs). Providing audio-visual summaries is also an important innovation here.

Fig.3 Dashboard summarizing contact-center performance trends in terms of KPIs, emotion and behavioral analytics (including outlier campaigns)

⁶ “Toward Detecting Emotions in Spoken Dialogs.” Chul-Min Lee & Shrikanth Narayanan. IEEE Transactions on Speech and Audio Processing, vol. 13, no. 2, pp. 293-303, 2005.
⁷ “Rate my therapist”: automated detection of empathy in drug and alcohol counseling via speech and language processing, B.Xiao et al, December 2015.
⁸ “An emotional connection matters more than customer satisfaction”, A. Zorfas and D. Leemon, Harvard Business Review, August 2016.
⁹ “The new science of customer emotions”, Harvard Business Review, November 2015.