As an analyst, I create a lot of reports. Thankfully though, I work with some great colleagues and clients who understand the value of solid metrics and continuous improvement. However, from time to time, I do still see benchmarks or references to industry averages in reports.
Here are the top three reasons why benchmarks shouldn't be used:
Benchmarks Limit Growth
Based on principle, the biggest issue with benchmarks is they compare you to the average, not to your potential. Averages are mediocre; the status quo, dull, mundane, the… average. And being average isn't something any organization or individual in the marketing or analytics industry should strive to be. If Amazon or eBay are dominating their industry's benchmarks, should they stop innovating and improving?
Take a look at your organization's "About Us" or "What We Do" pages - do you see the word average? No, you likely see "best," "leading," "innovative," or some other superlative that the organization strives to be. When an organization's performance is compared against the average, that's the day an organization starts to lose to its competitors.
Benchmark Data Isn't Accurate
Another large issue with benchmarks is that you often deal with fairly small percentages for KPIs, such as conversion rate. Even with the assumption that every organization in every industry were to provide their conversion rate metrics to a defining body on conversions (let's call it the DBOC) to establish more complete industry benchmarks, there would be inconsistencies with what constitutes as a conversion. For example, conversion rates could be based off of visits vs. unique visitors, offline and online vs. solely online, you get the picture.
Let's dig into this issue a little deeper. Going back to the issue of small numbers, let's assume that the benchmark conversion rate for your industry is 5 percent. Five percent of all unique visitors who visit a website in your industry complete a conversion. But, what's a conversion in the first place? Is it contact form completions? Contact forms plus newsletter signups? Job inquiries? You can see how each site in an industry can vary greatly depending on what they view as a conversion, potentially skewing benchmark metrics up or down.
A second issue with the accuracy of benchmark data is that the averages tend not to be weighted based on the size or volume of transactions. If benchmarks for e-commerce sites were to be accurate, the total transaction and visitor counts for every e-commerce site would need to be summed and then divided to take an average (the same could be said for B2B or lead-gen sites). For example (and keeping the numbers simple), let's assume that the e-commerce industry has four companies: Amazon, eBay, Best Buy, and Zappos (their hypothetical conversion rate and market share metrics are included in the table below).
|Company||Conversion Rate||Market Share|
By a standard benchmark report, the e-commerce industry benchmark for conversion rate would be 7.13 percent. However, when taking volume of sales (for sake of simplicity, let's assume this to be market share) into account, the conversion rate becomes 14 percent higher, or 8.13 percent. At first glance, that may not seem like that big of a difference, but when you think about what it would mean for Amazon to raise its conversion rate by 14 percent, that's a very significant amount of money.
By omitting the data that leads to more relevant results, standard benchmark reporting can lead you to believe you're doing better or worse than an average that likely isn't even accurate in the first place.
Benchmarks Don't Represent the Total Picture
The last issue on benchmarks I'll touch on is completeness. Benchmarks alone don't provide much insight; they lack context. If you're in a declining industry and most everyone has negative cash flow, is it OK for you to lose money month after month as well, as long as you're not losing as much as your competitors? Great metrics should be able to stand on their own, they should be straightforward, and they should help further the cause of improving results.
So What Should Organizations Do Instead?
Now that we've covered why you should throw industry benchmarks out, what you should focus on moving forward is actually pretty simple:
Great companies like Google, Apple, or Amazon grew to the giants they are by a constant desire to be even better than yesterday, never to be average.
Image on home page via Shutterstock.
Upcoming Webinar: PPC Pause and Reflections for 2013
Thursday, December 12 - 2013 was a major turning point in search advertising. With Google's Enhanced Campaigns and Bing's innovative Smart Search capabilities in Windows 8.1, now is a great time to pause, reflect, and plan for the new year.
Webinar attendance is free. Sign up today!
Robert Miller is a Senior Analyst at Search Discovery. He is actively involved in industry organizations, such as the Analysis Exchange and the Digital Analytics Association.
With the Analysis Exchange, he helps non-profits capitalize on their website data, and educates aspiring digital analysts about the foundation of digital analytics, from the implementation of a digital analytics tool to performing analysis and making data-driven recommendations for organizations.
December 12, 2013
1:00pm ET / 10:00am PT