Don’t measure the wrong metrics.
getty
Artificial intelligence (AI) has a CX measurement problem. It’s not that you can’t measure success it’s that many companies are measuring the wrong things.
A new study from Laivly, a pioneer in applied AI for the contact center, found that 65% of organizations considered their AI initiatives to be successful. That may seem like a successful number until you dig deeper into the data, which tells a different story. I had a chance to interview Jeff Fettes, founder and CEO of Laivly, for an episode of Amazing Business Radio, and he shared the danger in measuring the wrong metrics. My first take on his comments was that the wrong data will give you a false sense of security.
It turns out that 43% of AI projects don’t meet time deadlines, and more than half have exceeded their original budgets. And this finding summed it up:
Nearly three in 10 leaders (28%) said AI had directly contributed to lost revenue because it could not effectively handle complicated customer support issues, thereby frustrating customers and increasing churn.
In addition, one in five leaders (20%) said they know there is lost revenue but can’t quantify the damage.
While my focus is typically on customer service and CX, these findings go beyond the contact center. They will especially resonate with C-Suite decision-makers under pressure to deliver successful AI projects. In fact, Laivly found that 43% of boards of directors are dissatisfied with AI progress and are pushing the C-Suite to prove its success.
Defining the wrong success criteria is a major problem for any company implementing AI solutions intended to streamline processes and reduce costs.
Don’t Confuse Deployment with Success
Fettes says, “Organizations often celebrate milestones that are easy to see, such as implementation dates, reduced staff, chatbot launches, or how many tools are being deployed across the business. These metrics create the appearance of progress, but appearances can be deceiving.”
The Laivly research found that nearly half of organizations reported increased customer friction due to problems with their AI customer support solutions. Rather than making customer support easier, customers experienced more friction, including more transfers and having to start over and repeat their story. The result is increased frustration, which opens the door for competitors to steal customers.
Technology Doesn’t Create Value When It’s Deployed Unless It Also Improves Outcomes
The difference between success and failure when deploying any technology solution, not just AI, is an improved outcome. While this makes complete sense, some companies become enamored with the promise of what AI can do and “over AI” the customer experience.
Just because a higher percentage of calls are handled by automation doesn’t mean customers are happier. If they experience friction and feel the need to speak with a live agent, the automation isn’t working. The goal should be to improve the customer experience, not just to process more calls through chatbots or automation. Fettes says to always keep the customers’ feelings and needs at the center of all decisions.
The big mistake is when companies go “all in” on AI, only to find that replacing live agents with technology, while looking good on paper, causes higher churn. Eventually, many of these companies scale back AI, bring back their live support, and attempt to reduce churn and hopefully win back some of the lost business. Fettes was clear that AI should not fully replace human agents.
The Emperor Has No Clothes
While a failed AI implementation may seem to be the biggest fear, what leaders should be more concerned about is believing they have achieved AI success. This can be compared to Hans Christian Andersen’s famous fairy tale, The Emperor’s New Clothes, in which the emperor believed he was wearing magnificent clothes, and everyone around him reinforced the “illusion.” In reality, there were no clothes at all. But no one wanted to be the one who challenged him.
Many organizations approach AI the same way. Fettes says, “Leaders celebrate with milestones related to implementation, versus proof of progress or success.” That results in leaders over-believing in their AI solutions, where some capabilities are overpromised and underdelivered. Companies fall into the danger zone when they release a flawed system to the applause of everyone around them (inside the company). And the only ones giving them real feedback are their customers, who demonstrate that feedback by moving their business to the competition.
Final Words
The danger isn’t that AI fails. The danger is that companies prematurely convince themselves that they have succeeded. Like the emperor’s invisible clothes, the illusion can persist until someone, often the customer, points out the obvious.
None of this should suggest we should stop pushing to create a better experience using AI. It just means to recognize what works and what doesn’t. Success criteria shouldn’t be that we’ve implemented an AI solution. That success should be based on feedback from those using it, in this case, a customer. The biggest AI risk isn’t failure. It’s the illusion of success.
