The Web Site Law of Averages

A lot has been written recently in ClickZ and other publications about key performance indicator (KPI) development. Increasingly, businesses do the right thing and align metrics to Web channel goals and objectives. They ensure they measure the right things, the right way.

When you develop your KPIs or look at other performance measures, you may consider such metrics as:

  • Average time spent on site per visit
  • Average order value
  • Average number of pages viewed per visit
  • Average number of visits per visitor

These metrics are typically found in most Web analytic systems’ reports. Trouble is, they can be highly misleading. The majority of Web analytic systems report on these metrics using what’s know as the arithmetic mean, which is inappropriate for the type of behavior we observe on most sites.

For example, the average number of pages viewed per visit is calculated as the total number of site pages viewed divided by the number of site visits. The underlying assumption about using the arithmetic mean is the data has a normal distribution to it: the famous bell curve.

If the distribution of the number of pages viewed per visit is normal, around half the visits would be less than the average and half would be more. The majority of visits would be around the average value.

The reality is Web user behavior isn’t normal in the statistical sense of the word. It’s usually highly abnormal and highly skewed. Most visitors on most sites don’t often do anything of any value.

The chart below shows a fairly typical example of what you find on most sites. Here, we see the distribution of the number of visits (%) by the duration of each site visit (minutes). A similar pattern emerges for the distribution of the number of pages viewed per visit, the number of visits per visitor, the number of orders placed per customer, and so on.

Mason Chart
Click on graphic to view chart

In this case, the reported average time spent on the site is 6.5 minutes. But the distribution of visits is highly skewed. A very large number of visitors spent a very short time on the site, and a small (but significant) number of visitors spent an extremely long time on the site.

The net result is visit time averages to about 6.5 minutes. The reality is very different:

  • 50 percent of all visits actually last for three minutes or less (i.e., about half of the average).
  • 70 percent of all visits last less than 6.5 minutes.

Using these averages to report on the business obscures site visitors’ true behavior. These averages also tend to overestimate what the bulk of site visitors do in terms of time spent, pages viewed, and so on. The mean is also sensitive to behavior changes at the extreme right side of the scale.

What to do? An alternative is to use the median, the point in the distribution where 50 percent of people are (in our example, about three minutes). Unfortunately, most Web analytics systems don’t produce this metric. You must calculate it manually. It would be great if Web analytics vendors would provide this metric as a standard. WebTrends does report on the median visit duration as standard; users should look at how different the median is from the average.

We also advise clients to use a threshold measurement instead of an average and to look at the proportion of visitors who spend more than a certain amount of time on the site or who visit more than a certain number of pages. This threshold is usually easier to calculate than a median and perhaps is conceptually easier to understand for most business users.

What thresholds should be depends on the type of site and should tie to site goals. If it takes a minimum of three pages to reach any valuable content on a corporate Web site, for example, an appropriate metric might be the number of visitors who look at three or more pages.

Beware of averages. Look at the underlying behavior patterns on your site to get a better understanding of what’s really going on.

Related reading