As this is my last column of the year, I get to go early on the predictions for what’s going to be hot in 2013, and my view is that we’re going to be hearing a lot more about predictive analytics in the digital marketing space. My sense is that there are a number of factors that will lead to greater adoption of these analytical techniques among organizations over the next 12 to 18 months. Costs of data storage and processing are dropping, software is becoming cheaper and easier to use, and frankly, we all need better answers to age-old problems like marketing spend allocation and customer experience optimization.
Mind you, it’s been a while coming. Data mining and predictive analytical techniques have been around for years in brand and direct marketing analytics. And I do worry slightly that as these techniques become more widely used in digital marketing, there will be a zeitgeist effect as technology vendors and consulting organizations rush to jump on the bandwagon, saying things like “We do predictive analytics.” That’s a bit like saying “We do long division.” You’ve got to ask yourself what is the purpose of using data mining and predictive analytical techniques? What problem are you trying to solve?
For each type of analytical problem, there’s usually a range of techniques that can be used to solve that problem, of which data mining and predictive techniques are one class. Take segmentation as an example. The analytical problem in segmentation is to find groups of people (visitors, customers, etc.) who have something in common and also finding what it is that they have in common that’s different from other groups of people. The marketing or business problem you’re trying to solve is to improve the efficiency and effectiveness of marketing expenditure through increased relevance and personalization.
The analytical techniques available in segmentation range from relatively simple “deterministic” approaches, like creating filters in the data, through to more complex “data mining” techniques such as cluster analysis or neural networks. Web analytics tools today allow you to filter the data to create different visitor segments, and some do it better, faster, and smarter than others. More complex techniques such as cluster analysis are generally the realm of specialist statistical analysis packages such as SAS or SPSS, but there is growing adoption of solutions based on the open-source language “R,” or algorithms that are embedded directly into database environments such as Microsoft SQL Server.
For segmentation there are some clear advantages to using “discovery” techniques rather than “deterministic” techniques. Generally they are more powerful at identifying potentially interesting visitor customer segments that would be difficult to detect using conventional “slicing and dicing” of the data, no matter how good your package is. From my experience, valuable segments tend to be quite small and therefore hard to find. However, there is also a clear cost to using data mining and predictive analytical techniques. There are potentially hardware costs, software costs, process costs, and skills costs.
A lot of these explicit costs are coming down, such as the costs of hardware and software, but the process costs and skills costs are largely hidden. The cost of extracting, transforming, and loading digital data into a system ready for analysis can be quite significant. There are some real complexities in digital data that aren’t typically found in other data sources such as transactional customer data. Many organizations have struggled to tame digital data and incorporate it into data warehouses. It’s not that it’s impossible to do, you just need to know what you’re doing.
The skills costs are also a real cost to be considered. Typically statistical analysis skills aren’t often found in web analytics teams, mainly because there hasn’t been a requirement for them before. However, there will be a requirement going forward for organizations to have people who understand digital data and also understand the use (and often abuse) of data mining and predictive analytical techniques. Even if you’re using the results of pre-packed predictive analytics such as automated forecasting algorithms or outlier detection, someone needs to understand how the results are being generated and the limitations of the algorithms being used.
But don’t get me wrong, I’m not down on data mining and predictive analytics approaches, I’ve been using them for years (decades even). But as it becomes the next big thing (after big data), all I’m saying is caveat emptor. Beware of what you’re buying, what you’re buying it for, and how you’re going to be using it. Beware of shiny baubles and fancy packages and remember that predictive analytics and data mining is a means to an end and not an end in its own right. Don’t do predictive; use predictive to solve particular problems.
Fortune Cookie image on home page via Shutterstock.