The Trouble With Audience Metrics

Imagine that you’re asked to plan a campaign for an automotive manufacturer. This is a client that is new to your agency, and expectations are high on both sides. The automaker lets you know that its primary objective is targeting women who are in the middle of the purchasing cycle.

Fortunately, you know of a great site that you’ve been eager to use for the first time. You visit your Internet audience measurement service of choice to gather the metrics for your plan. But something doesn’t add up. This automotive research site that specifically caters to women is showing an audience that’s 60 percent single male.

If you happen to be new to planning and buying, such an experience might be enough incentive to omit the site from your campaign. You might take the data at face value, despite the fact that it’s a case of erroneous metrics. You could be shirking what amounts to the most effective site for your client (and cheating a perfectly qualified publisher out of a campaign) — all because you put all of your faith in one measurement source.

The Long Road to Gaining Deeper Insight

The accuracy of the audience measurement services that most advertisers rely on (primarily comScore and Nielsen//NetRatings, or NNR) is an issue that’s always lingering at the back of our industry’s collective mind as it relates to both site traffic and audience profiles. Now and then, it’s brought to light through the media. In 2006, a trade article cited major discrepancies (up to 30 percent) between publishers’ own internal data and that of the metrics suppliers. It incited comScore’s CEO to publish an open letter addressing the allegations.

The next year, the Interactive Advertising Bureau challenged comScore’s and Nielsen//NetRatings’ numbers through an open letter that asked the companies to agree to a third-party audit of their measurement processes. ComScore had the Advertising Research Foundation (ARF) produce an in-depth evaluation of its methodologies. The result was quite flattering, but prior to its release the IAB was quoted as saying such an audit “doesn’t include any kind of testing or evaluation or enforcement.”

Ultimately, both comScore and NNR agreed to enter into an audit conducted by the Media Rating Council (MRC). Last October, the MRC was quoted as saying it was still not at the halfway point of completing the process for either company. And so the effort to uncover the inner workings of the services on which countless buyers and planners stake the success of their campaigns continues.

At the heart of this discussion and the conflict with which these parties have been embroiled is the way Internet measurement services like comScore and NNR gather their data. Both comScore and Nielsen rely on panel-based research, whereby consumers are continuously monitored as they surf the Web. This approach has its benefits. Unlike metrics determined by internal server log data, for example, panel data counts people and not cookies (which, of course, are not one and the same). This factor — along with cookie deletion and U.S. versus international traffic — remains one of the leading explanations of the common metrics discrepancies that continue to exist.

Back to that automotive research site for women: unfortunately this example wasn’t fiction. Nor is it the only case of a publisher’s audience metrics being far from harmonious with its internal numbers. Many a publisher will share a similar tale, and many buyers and planners have seen the discrepancies for themselves. Problem is, they may not have realized it.

What, then, is a buyer to do, and whose numbers should she trust with her client’s campaign? The answer might lie with the publishers. In an effort to ensure metrics and audience profile accuracy, many are looking to additional sources to analyze and measure their visitors, and to produce reports they can offer to those advertisers who might question them. At the same time, other third-party measurement services that take a different approach to harvesting their data can be looked to for a second opinion. Next week, I’ll address both of these trends and how they stand to improve our campaigns.

Related reading

Vector graphic of a megaphone spewing out business themed items, such as a laptop, tablet, pen, @ symbol and smartphone