When Is a Visitor not a Visitor?

Controversy surrounding unique visitors, a core metric of Web analytics, has raised its head again. In a recent blog post, Web analytics consultant and author Eric Peterson called on the Web analytics industry to stop using the term “unique visitors” because it doesn’t accurately reflect what’s actually being measured.

The issue stems from the Interactive Advertising Bureau’s proposed definitions on audience reach metrics, including unique visitors. The IAB’s definition of unique visitors differs from the Web analytics industry’s commonly used definition — which is cookie based.

I agree with Peterson’s assessment (if not entirely with his style and approach) that the unique-visitor metric from a Web analytics tool is potentially very misleading.

Keep in mind, Web analytics tools define unique visitors fundamentally different from an audience panel such as comScore and Nielsen/NetRatings. This is highly confusing for many people. For publishers and other sites that depend on being able to demonstrate the size of their audiences, it’s a bit of a nightmare.

Web analytics systems define a “unique visitor” based on the presence of a cookie. If I visit a site using three different devices in a week (say a PC, a laptop, and a mobile phone), I’ll be recorded as three different “unique visitors.” If I also regularly delete the cookie, then I can appear to be a new visitor to the site and am therefore not counted as being “unique.”

Audience measurement panels define a unique visitor based on the activity of an individual member of the panel. Web analytics tools measure all activity on a Web site (a so-called census-based approach), whereas a panel measures a proportion of the activity on a site (a sampling approach) and then uses that to estimate the total.

So with two different definitions and two different data collection methodologies, it’s hardly surprising that people can get confused and debates rage about which numbers are right and wrong. It would certainly be helpful if Web analytics tools came up with another name for unique visitors. That’s a massive issue for the industry to address. Hopefully, it can be done collaboratively.

Don’t count on it getting resolved any time soon, though. For site-centric measurement, cookie-based tracking is the closest that we get to the notion of a visitor. There are some specific instances in which it becomes possible to link cookies to real people to get a proper unique visitor count, but it generally involves massive amounts of processing and massive amounts of sensitivity to privacy issues.

However, this debate reinforces the need for “information consumers” to understand how the data they use for business decisions are put together. Therefore, the Web analytics industry must continue to educate users of this data on its “provenance,” warts and all.

One of my big concerns about the unique visitor metric from a Web analytics tool doesn’t involve its “technical efficacy” but more the messages it sends out. The implication from reporting on visitors from a Web analytics tool is that you can fully understand your Web site visitors using clickstream behavioral data.

Because we call this metric “unique visitors,” it lulls us into a false sense of security that we’re actually tracking “visitors” rather than “devices,” and therefore don’t really need to understand anything else. However, clickstream data doesn’t tell you anything about who is visiting the site, why they’re there, and what they thought about the experience.

In “My Simple Organization Maturity Model,” I talked about customer centricity. Customer centricity involves having a 360-degree view on your customers and being able to answer the “who” and “why” questions, as well as “what” and “when.” Web analytics are just one tool in the toolbox. They’re absolutely necessary, but rarely sufficient.

So, the unique visitor debate will go on. But if the discussion reminds us of what we’re measuring and how we’re measuring it, then it’s a useful debate to continue.

Disclosure: Neil is a board member of the Web Analytics Association (WAA), the industry association concerned with setting standards and definitions of measurement in Web analytics. The views expressed here are Neil’s personal views and are not, in any way, an official view of the WAA.