Home  › Marketing › Strategies

Your Marketing Data Sucks

  |  August 13, 2010   |  Comments

With online advertising and marketing, I think we need to ask the single most important question about research: why are we bothering?

The Interactive Advertising Bureau - the single most consistent and important source of guidance for our industry - has just released a study into an advertising research method that many (all?) of us have used at some point: website intercepts. Its finding? Not so good.

Ends up that the practice of catching people on the Web and asking their opinions about brands, products, and (most importantly) ads has some serious flaws. In fact, the report points out three problems that are most prevalent and serious:

  • Low response rates: This is clearly an issue that can kill any real research. If you don’t get enough people to talk to, you can’t make any reasonable claims. When you see a piece of research, such as a poll, for example, there is always a margin of errors. This is done through a fairly simple calculation that requires the population size (the total number of people you are talking about) and the sample size (the total number of people you actually talked to). You can figure this number by yourself, or with any of the many online calculators available. The population size is fixed, but the sample size is dependent upon how successful you are at reaching people; if you are unsuccessful, than the margin of error is huge, and your study is worthless.

  • Untested research methodologies: This issue is the subject of debate, especially in some forums that focus on advertising data. While there are clearly well-established methods for performing research, that isn't necessarily a static list. The IAB is being a bit of a stickler for tradition in this case, which is certainly the safe thing to do. But methodology - the way in which you ask questions and analyze response - is really a matter of understanding and clarity. I like to apply the common-person rule: if the methodology seems reasonable to a reasonable person, then it can be acceptable.

  • Improper weighting: Weighting is a key analytic task that gets farther into the art than the science of numbers. Certain responses and respondents are more important than others. Also, some key data points need to be given more of a representation in an overall sample. Determining which points those are and how much you should weight them is essentially taking something that we know (women know more than men about household finances) and putting it into the math formula. But that weighting needs to come from something provable and rock solid. The weighting going on, in the eyes of the IAB, isn’t that.

These are certainly not problems that are unique to online advertising research. In fact, many studies, academic, political, and otherwise, fall prey to these precise issues.

But, with online advertising and marketing, I think we need to ask the single most important question about research: why are we bothering?

Hunter Culture and Farming Culture

A key moment in the development of modern society was when humans began to hunt for their food less frequently, instead growing it in their own backyards. We’re on the cusp of experiencing a similar shift in advertising research. Over time, we should become less reliant on surveys and other data hunting tools and more reliant upon capturing and understanding the data that's within our own (digital) backyards.

All of the problems related to research methods stem from the fact that we - as researchers - were not able to directly observe the consumer engaging with the product (or the advertisement of that product), nor were we able to understand anything more about that person, solely from their connection with our brand.

But very rapidly, that's becoming untrue. We are overwhelmed by data today about the consumer. We can directly observe their engagement with our advertising, we can see actions they take after seeing that ad, and we can learn tons about them through social networks. As interactive technology continues to spread throughout our lives thanks to super-powerful mobile tools, that set of data is going to grow and grow.

Why, then, would we invest large amounts of time in hunting for more data? Why would we invest in hunting for more data that potentially has problems?

The simple answer is that we still don’t quite know what to do with all this data, let alone how to do it. Plus, all of this data tends to reside on different servers, in different formats. Blending data sets together today feels a lot like mixing oil and water.

We are, however, on a path that's leading us to a more comfortable place where we can be much more effective with this data. The platforms are slowly moving toward one another and, especially as they consolidate, we all have hopes they will get better and the data will be more flexible and usable.

I truly hope the IAB is considering this, as it goes deeper into this issue. One of the action items it has identified out of this research project is the formation of a task force to tackle the problem. But I hope it isn't going to solely focus on how to get intercepts to work better, but rather look at how to better understand consumers.

Otherwise, I’m afraid we'll simply have recommendations for improving the buggy whip.


ClickZ Live New York Want to learn more?
Attend ClickZ Live New York March 30 - April 1. With over 15 years' experience delivering industry-leading events, ClickZ Live brings together over 60 expert speakers to offer an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!


Gary Stein

Gary Stein is SVP, strategy and planning in iCrossing's San Francisco office. He has been working in marketing for more than a decade. Gary lives in San Francisco with his family. Follow him on Twitter: @garyst3in. The opinions expressed in Gary's columns are his alone.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Marketing newsletter delivered to you. Subscribe today!




Featured White Papers

A Buyer's Guide to Affiliate Management Software

A Buyer's Guide to Affiliate Management Software
Manage your performance marketing with the right solution. Choose a platform that will mutually empower advertisers and media partners!

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.


    • Website Optimizer - SEO, CRO, Analytics
      Website Optimizer - SEO, CRO, Analytics (Marcel Digital) - ChicagoMarcel Digital, an award winning interactive marketing agency established in 2003...
    • Director of Marketing
      Director of Marketing (Patron Technology) - New YorkDirector of Marketing We are seeking a Director of Marketing to manage and build our marketing...
    • Senior Interactive Producer
      Senior Interactive Producer (Ready Set Rocket) - New YorkWhat You'll Do As a member of our team, the Senior Producer reports directly to our...