AnalyticsAnalyzing Customer DataData Integration, Part 1: Macro Integration

Data Integration, Part 1: Macro Integration

What's the best way to integrate Web analytics data with other data forms? First in a two-part series.

During the session on multichannel metrics at Search Engine Strategies (SES) in New York a few weeks ago, we had a question about integrating Web analytics data with other forms of data. What’s the best way to do this? As is often the case with Web analytics, the answer is usually, “It depends what you’re trying to do.”

One challenge digital marketers typically face is the number of different data sources they deal with at any one point in time. These might include data from:

  • Ad-serving systems
  • PPC (define) bid management systems
  • Web analytics systems
  • Affiliate management systems
  • Transactional systems
  • Customer management systems

The list can goes on. How do you pull all this stuff together to get a sense of what’s really going on?

I tend to think about data integration on two levels: macro integration and micro integration.

Macro integration is the process of pulling the outputs of the various data systems together into one place so you can see relationships between different data sources. Excel is often the ultimate macro data integration tool. Legions of analysts all over the country spend hours pulling together spreadsheets from multiple data sources to provide a summary of what’s going on.

This isn’t necessarily a bad thing. Pulling data together means the analyst is at least looking at them and is likely to spot any data quality problems. Problem is, 90 percent of the analyst’s time is spent on data production. Only 10 percent is spent on data analysis. The value equation is all wrong.

One approach is to try to automate the process as much as possible and get the various data sources into a single database. The data are likely to be processed or summarized rather than displayed in their original format, but at least they’re together which makes reporting out across the range of key performance indicators (KPIs) and key metrics easier. Inevitably, this requires some upfront investment to set up the data feeds and reporting system correctly. Standard databases and business-intelligence reporting tools could be used for this type of work.

At a company I once worked at, we took this approach to reporting on our key metrics: We extracted summarized visit, unique visitor, and page view data from our Web analytics system and stored it alongside other key company data in a data mart specifically designed for that purpose. The extracts were done nightly, so every morning the key site metrics were available and reported against some of our other key transactional data using a business intelligence tool (in our case, business objects).

Following the inevitable initial teething problems, the process worked quite well. Questions from management and colleagues changed from when the reports were going to be ready to what the reports were actually telling them in terms of business impact. We were able to go from spending 90 percent of an analyst’s time on data production to 90 percent of an analyst’s time on…analysis!

Obviously, this comes with a cost. Managing multiple data sources is a messy business. Significant resources may be required from your IT team, and they may or may not have the necessary experience dealing with this particular data type. An alternative route may be to outsource the process to such companies as BlackFoot Inc. that specialize in this type of data integration activity.

Next, I’ll look at micro integration — the challenges and opportunities in pulling together and integrating data at the customer level.

Till then…

Related Articles

6 ways to increase your conversion rate using behavioral data

Analyzing Customer Data 6 ways to increase your conversion rate using behavioral data

4w Mike O'Brien
Influencer marketing: Eight tools to identify, track and analyze your brand's next biggest fan

Content Influencer marketing: Eight tools to identify, track and analyze your brand's next biggest fan

4w Tereza Litsa
Tools and tips for calculating the ROI of social media

Conversion & ROI Tools and tips for calculating the ROI of social media

1m Clark Boyd
How to use behavioral data to enhance your website's conversion rate

Analytics How to use behavioral data to enhance your website's conversion rate

2m Chris Camps
Big data in the travel industry: How can travel companies do more to collect and use customer data?

Analyzing Customer Data Big data in the travel industry: How can travel companies do more to collect and use customer data?

2m Clark Boyd
How machine learning can help you optimize your website's UX

AI How machine learning can help you optimize your website's UX

1m Chris Camps
Why banks are becoming customer-centric organizations

Analyzing Customer Data Why banks are becoming customer-centric organizations

1m Al Roberts
How to achieve true omnichannel relevance

Analyzing Customer Data How to achieve true omnichannel relevance

2m Clark Boyd