I have a love-hate relationship with KPIs (key performance indicators). When done right, they are mission-critical to defining success and can focus the organization on the right priorities. When chosen poorly, KPIs can be disconnected with ground realities and be a constant source of frustration for team members trying to impact them.
However, poorly designed KPIs are not my primary gripe, at least not in this post. My main concern is that even well-designed KPIs are simply not deeply insightful, but they are often used as if they are.
Well-designed KPIs are full of contradictions. On the one hand, they are expected to be simple, easy to communicate, and intuitive. On the other hand, they’re expected to provide actionable information and be a reliable measure of important success criteria.
Anyone who has worked on developing KPIs knows that it is a game of balance and compromise, based on business objectives. The need for actionable information battles with the desire for simple metrics. The desire for intuitive metrics battles metrics that push status quo thinking or properly reflect the diversity of business interactions.
After many years working with and helping clients identify KPIs, I have found ways to manage their dichotomous nature, but never overcome it. If there is a brilliant mind out there who has solved for this, I would love to hear about it. For now, I will assume that this dichotomous struggle is a law of nature.
This leads me back to my main point. Marketers need to stop viewing KPIs as major source of insights. They are, as the name illustrates, only “indicators.” While this seems like an obvious statement, it is surprising how often KPIs have become the primary source of insights for most companies.
Take digital analytics, for example. Most companies using the web analytics platform use default metrics, such as clickthrough and page views, as their primary measures to understand web activity. While these metrics may indicate increased interest in content, they rarely tell you how satisfied the visitor was with the content or how valuable it was in decision-making. It is rare for companies to set up custom metrics and reporting, which might provide better insights. It is even rarer for companies to download raw web data into a data management tool and truly analyze visitor interaction with content, even though these solutions exist. Instead, most companies use the default web KPIs to derive custom insights into behavior on their website.
Another example can be found about how companies use social channel data. There are some great social analytics tools out there. When I come across most implementations, however, they are mostly set to track high-level sentiment analysis and rarely deliver deep insights. However, the underlying data is often volumes of highly informative, unprompted, free-form feedback. It has the added benefit of being free of interviewer bias or agenda-setting.
Recently, I was working on a project for a client that viewed their products as very innovative. Yet, when mining nearly 1,000 instances of social data, we found only one unprompted mention of innovation. Upon further investigation, we found that innovation was meaningless to the consumer. Instead, it was performance, excitement, and fun that consumers talked about most often. The customer was conveying what innovation really meant to them, while the company was still thinking in terms of engineering sophistication. This insight was un-minable from the standard social KPIs. Even traditional survey-based market research may not have captured this insight, as it would have relied on coming up with the right questions to uncover this disconnect between the company and its customers.
These examples demonstrate the need to dig deeper for better insights and I risk the label of “Captain Obvious” by making this assertion.
So, let me add to this. Well-designed KPIs, because of their simplicity and action orientation, often lull us into overestimating their insightfulness. This link is unconscious and habitual.
When I have asked marketers “What is your (Social, Web or Customer) data telling you?” A common response is, the (relevant) tracker is telling us [fill in the blank].
In reality, the answer to the question is rarely found in the tracker or KPIs. Even if they can point to a KPI that is helpful, the underlying explanation is still often conjecture or a hypothesis. In fact, the better aligned the KPI story is with commonly accepted wisdom, the more likely it is to be seen as data-driven thinking.
In other words, we find an interesting KPI trend and create a believable story around this trend and that becomes data-driven thinking when it is still just conjecture. It takes great discipline to put on the brakes and look for deeper and corroborating evidence and that is what KPIs really calls for.
“The better aligned the KPI story is with commonly accepted wisdom, the more likely it is to be seen as data-driven thinking.”
I want to make clear that this post is not advocating for the elimination of KPIs. They are very helpful tools for aligning the organization and most of us understand that they are only indicators. When done well, however, they are insidiously brilliant at creating the illusion of deep insight; especially if the resulting story is a good one. Truly data-driven marketers should be aware of this and be ready to dig deeper before letting a KPI drive strategic decisions.
