In my previous ClickZ column from 2009, “What’s Wrong With the Net Promoter Score,” I talked about the many things that were and still are problematic about the Net Promoter Score (NPS). Proponents argued back that it has been widely used for a long time. But that’s like arguing that both science and the church believed the Earth was flat for a long time – that is, until they discovered that was completely and utterly wrong. In this column, I will propose a few alternative metrics and techniques that marketers can use to truly tell if they will be able to increase sales.
But let’s first review some of the shortcomings of the NPS.
Net Promoter Score – Bad Math, Lagging Indicator, Not Actionable
Yes, I could be more direct. The Net Promoter Score is based on bad math. Many scientific studies have shown the error of asking a “uni-polar” question – “how likely are you to recommend?” and then applying a “bi-polar” calculation – “% promoters minus % detractors.” It also begs the question why the scale is not something symmetric like -3, -2, -1, 0, 1, 2, 3, instead of 0-6 detractors, 7-8, passives, and 9-10 promoters. Also, the supposed insights are gathered by asking survey participants to say “why” they gave the particular score. This has the same problems as surveys that ask a statistically insignificant number of users what they think, instead of observing what a statistically significant number of users actually do – which is now possible through digital and social channels.
And sophisticated marketers have long realized the NPS is easy to game, simply by asking users to rate something slightly different. Also, two companies may have the same NPS score even if the underlying data is entirely different – for example, a company with a 20 NPS could have 20 percent promoters, 80 percent passives, and 0 percent detractors, while another company with the same 20 NPS could have 60, 0, and 40. Again the NPS doesn’t tell you why, and therefore it is much harder to know what to do. In fact, I think of the NPS as a “lagging indicator” because if it went up it may directionally tell you whatever you did was better, or vice versa. But it doesn’t tell you what you did worked and certainly doesn’t have predictive power. Finally, a low NPS for a bank doesn’t mean all its customers will defect. They may simply be staying with the bank because it’s hard to switch to another bank; in other words, there is no connection between NPS and causality. The low NPS doesn’t tell the bank what they need to improve. So, I guess I can chalk the historic usage of the NPS up to it being super simplistic, easy to understand, and therefore easy to communicate to upper management as some sort of “proof” that the marketing was working.
But, it’s time to get more sophisticated.
Search and Social Velocity Are Better Indicators
With digital and social channels, marketers are now firmly in the era of “big data.” But thus far, most marketers have approached it as “more data for targeting.” The bigger opportunity is to use the insights from these channels to see whether certain marketing activities or even changes to the business (e.g., pricing, revenue model, etc.) or innovations to the product are likely to lead to more sales. The key here is to observe what users actually do, rather than ask them what they think.
Did they search more? Because users’ habits have changed – they go online to research virtually everything – marketers can use this to gauge interest and demand. Even if awareness were stimulated in traditional channels like TV, print, and radio, search and website analytics will quickly reveal whether users searched more, and even exactly what they searched for. This will reveal what exactly worked – a new feature, new pricing, new color, etc. (seriously, try this search for “brown M&Ms” – 2012 was when it aired the “Just My Shell Super Bowl” commercial).
Did they share more? With social networks and users becoming more accustomed to sharing online, marketers have new places to observe this behavior. The conversations that used to happen around the water cooler are increasingly happening online. Users tend to share the things they like or the things they think their friends will like. And very often they don’t just share with the entire world, they will share specific things with specific friends, knowing what she would actually be interested in. And if a recommendation came from a friend, a user is much more likely to look at it and consider it than if it came from an advertiser. Exactly what they shared gives the marketer new insight about what was important to the user and to the person with whom it was shared.
Did they buy more? Ultimately, good marketing comes down to whether more users bought the product or service in question – or at least it should. In my client work I have seen a huge increase in the number of manufacturers “going direct” to consumers. Historically, the manufacturers that sold through retail partners hesitated to do their own e-commerce, for fear of taking sales away from them. Now they realize that by doing e-commerce and having a direct relationship with the end-consumer, they can gather new insights that may even enable them to better help their retail partners.
Did they review and advocate more? Finally, if the users bought the product, used it, and actually loved it enough, a number of them will come online and write reviews (see the “1-9-90 rule of social media participation“). Some of these reviews will even be thoughtful and well written, and helpful to the next set of users who are considering the specific product. This starts the virtuous cycle of advocacy and loyalty driving the awareness and consideration of new customers.
In conclusion, the actions of users that are observable in digital and social channels yield incredibly detailed insights about what they were interested in and actively looked for or shared. The velocity of the changes in metrics such as search volume, sharing intensity, and number of reviews written at the time some marketing program was in market or some product innovation or business model innovation was launched can yield direct correlation and attribution to the specific action taken. Using these insights, marketers will know what to do more of or what to do less of. So, they no longer need to rely on an outdated and over-simplistic fake number such as the Net Promoter Score.
Banished image on home page via Shutterstock.