Mining For Gold

Every time a web visitor clicks a link to see another page, several pieces of data are stored in your web server’s log files. This data can be used to build and maintain databases that provide valuable marketing opportunities.

The server logs include data on which page or graphic was served (the referring page or URL) as well as the date, time, and browser information. Personalized sites and sites using databases often include profile or session identification codes that also show up in the server logs.

This generates a mountain of data you can use. For a long time, though, the only analysis of this data consisted of counting page views by day, time, browser type, plus a few other metrics that are interesting but hard to use in decision-making.

On the other hand, for several years traditional direct marketers have used a set of analytical techniques called data mining. And the increasing amount of available data on the web is driving renewed interest in data mining, which is gaining in popularity.

In fact, International Data Corporation has recently projected that market demand for data mining tools will grow from the $259 million reported for last year to $1.78 billion by 2003!

Early data mining products were best used by experienced data mining consultants and analysts. For many situations, this is still the case, but some of the newer data mining tools now make it possible to tap into these analytical tools without having a Ph.D. in statistics or an understanding of how neural networks operate.

Even with the newer data mining tools, there is still a considerable amount of work required to become proficient in using them to identify meaningful results. However, as a site accumulates large amounts of traffic data and profile information, data mining is one of the few techniques that can handle these needs. But before launching off into data mining, it’s important to understand what it can do for you.

First, data mining is not a single technique, but a set of statistical techniques that is used to identify trends, patterns, and relationships in the data. Some techniques are used by analysts to explore data, while other techniques are somewhat automated and are used to spot patterns.

Most data mining tools can create several different mathematical models from data, but two models are especially valuable to marketers looking to understand the segments within their markets – classification and clustering.

Classification techniques assign people (or whatever the data represents) to classes determined by the analyst. A simple example would be to analyze data on customers and non-customers who had either visited the web site or not. This simple example could easily be represented by a 2 X 2 table, but as the number of data items increases, it quickly grows beyond simple tables.

Clustering techniques are used to identify occurrences in the database with similar characteristics, subsequently grouping them into clusters. Unlike classification, the analyst doesn’t specify the groupings ahead of time, and the results of clustering may or may not be valuable.

For example, it would be expected to find that people who buy an air purifier would buy replaceable filters, but what if customers are actually buying less expensive filters designed for a different product? In this case, a data mining analysis could point out an opportunity to promote the value of using the recommended filters over the cheaper filters.

For web marketers, the two types of data that really make data mining a terrific tool in identifying golden opportunities are the profile database and the web server’s log of page activity. With these two sources of data, you can glean an enormous number of insights.

For instance, you may suspect that visitors coming from two similar content sites actually have different personalities, and therefore, different purchasing behaviors.

You may also have wondered if the same customers who had purchased another product a few months earlier, purchase certain products. And, does this pattern hold true for all sections of the country?

Data mining can help you answer these questions by combining classification with another common data mining technique, time-series analysis. By analyzing customer data across products and across time, data mining can spot patterns that would be hard to identify with traditional database tabulations.

In addition to spotting cross-selling opportunities, data mining can also be used to:

  • Analyze customer acquisition and retention promotions over time.

  • Learn which combinations of products are purchased.
  • Identify meaningful market segments using profile and web activity data.

While it takes a special effort to build and maintain the necessary databases, the ability to spot golden nuggets of marketing opportunities within your web audience data can make this effort very profitable.

Related reading