Combining Web Analytics and Qualitative Insight
The why of analytics is as important as the' what.
Recently, my ClickZ colleague Neil Mason wrote a column on qualitative analysis as part of a series he’s writing on different data sources that help companies understand Web site performance. Neil outlines a number of great points; take a quick read if you missed it.
The column elicited a fair amount of response from people within the Web analytics space on Eric Peterson’s Web Analytics message board. Jim Novo made a really good point I want to build on. He argued that the way in which analytics data and qualitative data should be used together is to “get the VOC [voice of customer], but first understand the behavior so you have a concrete idea of whose ‘voice’ you are really hearing.” He went to say, “I don’t value the opinion of customers who say they will buy as much as I do the ones that did buy.”
Three Key Types of Data
We’ve had a lot of success using quantitative, qualitative, and competitive data to solve online problems and build site optimization solutions. Depending on the situation, we’ll use the data in very different ways and in different orders. In analyzing and optimizing site performance, we value all three data types but are careful in the ways we use them. I don’t believe just one of the three is the Holy Grail.
One method we use most often resembles Novo’s point. We start by understanding the customer’s overall site behavior. For a commerce site, we’ll ask questions such as: Are people buying? Where are they dropping out? Are they even getting to the product level to look at an individual product?
Identify and Monetize Opportunities
Once we define overall site goals and desired behaviors, we dig into the analytics information to begin identifying different improvement opportunities. Once we identify these opportunities, we rank them based on the monetized value of potential performance lifts. Starting with the issues with the greatest potential monetization, we often turn to the qualitative side of things, such as surveys, focus groups, and expert reviews. We try to really understand the why behind the issues identified.
We could just go right into optimization without the qualitative insight, but we would simply be guessing at the why. In some cases, we’ll do this based on obvious problems or when the analytics indicate fairly strongly the why. But without the qualitative analysis, there’s less chance of solving the problem through optimization. You may completely miss the mark on the why and optimize in a way that really doesn’t address it.
Treat the Ailment, Not the Symptoms
The danger here is you could actually improve a page’s performance according to the metrics but still not solve the problem that drives most people away. You may be satisfied with the lift and move onto something else on the site, leaving a lot of money on the table by not truly solving the problem that pushes most people away.
A 3 percent conversion rate for a specific process may seem good if it was at 2 percent before you made changes. But if you took the time to really understand through qualitative analysis methods why people weren’t completing the process, you may have identified an issue that would have raised the conversion rate to 6 percent. But again, you’re satisfied with the increase to 3 percent and move on to the next thing.
What’s Really Going On?
There’s a greater risk of misinterpreting the data and barking up the wrong tree all together if you don’t address why you’re getting certain results. You may be frustrated by people exiting the site from a product page instead of continuing to order the product online, for example. Though there are often many things you can do to optimize a product page, a true understanding of your customers may reveal they’re calling you to place the order.
Dig deeper, and you may find callers close at a much higher rate and place a larger than average order. You may also find those Web customers average a shorter call time. Depending on your business, degrees of lift in the higher close rate, average order size and lifetime value, and call center costs, keeping those calls coming in may be much better for the business. In this case, relying only on Web analytics could cost you some high-value customers.
On its own, analytics can identify many different issues, but integrating qualitative insight and other data types with the items identified through analytics can greatly increase the value of your Web analytics initiative. The why (qualitative) is just as important as the what (analytics).
