Choosing Your Perfect Software Match, Part 2

This is part two in a series based on my talk with Matthew Berk, senior analyst for Jupiter Research, which shares its parent company with ClickZ. He’s a former IT guy who is passionate about site analytics and empowering marketers with the tools to make good decisions.

In part one, we learned about “Berk’s list” — the list of software and solutions he personally would use to run his next Web site. The list is divided into six categories, and Berk’s promised to name names in each. We resume our conversation with Berk postponing the naming of names while he explains what he considers ground zero for operating a Web site.

Smith: So, are you going to name names for us?

Berk: Yes, I’ll name names. But first let me tell you what I consider ground zero for operating a Web site. You must have site analytics, end of story. And the solution you choose must be able to marry what I call “the 5 Ss”: source, stream/scenario, segment, self, and score. The vendor you choose should be able to help you correlate four out of five of these.

Smith: Explain.

Berk: Source is where your traffic came from — the marketing vehicle by which you acquired them, in most cases.

Stream refers to traditional clickstream behavior or scenario analysis — understanding how to optimize for attrition in a given process like registration or checkout.

Segment is the ability to refine your view of customer behavior to make it more meaningful. Aggregate conversion rate tells you nothing, but a segmented conversion rate helps you figure out which groups of customers are most valuable and why.

Self refers to self-reported data, usually collected in surveys but matched with how the user actually behaves. Ideally, you want to tie self-reported metrics like satisfaction to purchase behavior or usage of specific site functionality.

Finally, score is the ultimate outcome: where the money is. It could be average order size, top-line revenue, number of subscriptions, time spent gaming, or number of bills paid; it all depends on the business metrics which define the value of the site. The reason I say a vendor should be able to do four out of five of these is because most don’t include surveys — the self-reported data.

Smith: What about sites that aren’t e-commerce sites? For example, the Georgia Department of Transportation (DOT) has a site that gives traffic information so people can plan their commutes. There’s no revenue goal.

Berk: Sites with no commerce focus should still focus on the 5 Ss. For those, the score is best expressed as the rate of task accomplishment. What do users want to do on the site? What does the DOT want users to do on the site? Are those things being accomplished? Regardless, your analytics tools should focus on merging the 5 Ss so that you get a complete picture and can analyze specific scenarios to find opportunities to optimize the site.

Smith: So you said you’d name names…

Berk: I’ll name names. The list is segmented into the six categories I gave you. Start with site analytics, since it’s the most important. My list includes Omniture, WebSideStory, and NetIQ.

Smith: NetIQ — maker of WebTrends?

Berk: Yes. People are surprised to see NetIQ on there, but its product has really made some great strides in the past six months, whether you’re running a small site or a large, complex one. Still, Omniture is my top pick. I probably wouldn’t put Coremetrics on the list unless I was doing high-end consumer retail and could get the company to give me a real break on price.

Smith: What exactly does site analytics include? What are these tools supposed to do?

Berk: Site analytics is not, contrary to popular opinion, about reporting traffic data — or reporting anything else, for that matter. It’s all about helping you marry your metrics (the 5 Ss), so that you can discover opportunities to improve the site.

Smith: What’s next?

Berk: The next category is a survey tool. I might build this in-house. But also on the list are companies like BizRate, ForeSee Results, and Insight Express. You might choose a vendor if you either need to conduct many surveys or if you need a specific kind of survey (e.g., customer satisfaction). But it’s not too hard to build for the basics.

Smith: And it’s possible to use manual intervention to merge your survey data with your other customer data.

Berk: Next is a panel-based usability testing service. Vividence or Insight Express can help out here. Still, many of my clients fare wonderfully well with a simple usability lab. A few grand and two spare offices are all you need to start learning. To test new functionality, you really need feedback from actual users.

Once again, readers, I’ve reached my word limit. Next time we’ll learn Berk’s picks for the final three categories of Web measurement tools. We’ll also hear why he feels strongly that ASP solutions are the way to go.