Home  › Media › Media Planning

The Problem With Online Media Measurement Tools

  |  May 6, 2008   |  Comments

How savvy media buyers can mitigate discrepancies in measuring online audiences.

In late February 2008, Google's stock dropped 4 percent in one day after comScore released data suggesting the search giant's click-through rate on paid search ads was decelerating. comScore claimed that Google's growth rate had dropped from 37 percent in October '07 to 0.3 percent in January '08. Soon after, comScore spoke up, pointing out the drop off wasn't a sign of weakening, but instead due to Google's click quality programs. When Google released its 2008 first quarter earnings, it blew away all talk of weakening. Google reported strong click growth of 20 percent year-over-year and revenues of $5.2 billion, up 42 percent year-over-year.

This isn't the first time the accuracy of a ratings company's data has been called into question. For top publishers that live or die by numbers, ratings companies' methodologies often don't measure up. In September 2007, Spanish newspaper "El Pais" sued Nielsen Online for downgrading the number of unique visitors the site received. "El Pais" claimed the downgrade would result in a $1.4 million loss in ad sales because media buyers rely on panel-rating companies to make media buying decisions.

Problem is, no single measurement system is without flaws. The grey areas caused by relying on panel ratings, raw server data, tagged pages, downloaded software, and even public opinion almost ensure that wise media buyers will have only a fuzzy loyalty to the numbers.

Measurement Methodologies

Currently there are several ways that metrics, click rates, and traffic are gathered by ratings companies. Nielsen NetRatings, comScore, and Compete rely on panel-based measurements, whereby they extrapolate numbers from a "representative panel" of users. But, just how accurate is that representative pool for any given buy? Hitwise doesn't use panels, but instead analyses anonymous data provided by ISPs. But, how representative are the participating ISPs? A movement toward more "open" ratings systems has brought companies like Quantcast into the arena, whose method combines audience-measured data (publishers place tracking code on their pages and share their data) with panel-based estimates. But for non-participating sites, Quantcast's evaluation can be extremely off base. Companies like Alexa rely on numbers obtained from people who've downloaded and use their toolbar -- which in most cases tend to be online marketing people and skews the data.

In addition, publishers often employ their own analytics programs or server logs to provide data for advertisers on unique visitors and page views. The smaller the publisher, the more you'll want to probe to find out exactly what analytics program they're using to measure traffic. Countless other articles have been written analyzing the discrepancies among the data provided by these various measurement systems. What a mess when you think about it!

For all this inconsistency, bear in mind, too, that many measurement tools are expensive. Large agencies might buy licenses to more than one system, but few others can afford to.

Considerations for Media Buyers

Is any one measurement system better than another? How does a savvy buyer mitigate the discrepancies? How can these tools -- warts and all -- be best utilized by media buyers to influence purchasing decisions? What new measurement tools are on the horizon due to the Web 2.0 phenomenon?

The media buyer needs a place to start assessing a site -- a ratings tool -- but must retain the perspective that none of these measurements are perfect. We've seen about a 10 percent discrepancy among various ratings systems for the same sites. Media buyers must rely on comparative research and trained judgment. Breaking the addiction to numbers can also be liberating. Many niche sites don't even register on the ratings companies' radars, but they might be more qualified to be included in the strategy than a larger site with a "channel" buy. You may pay more for a smaller audience, but achieve better brand impact or conversion rate due to the relevance of the product/service to the audience.

Most media buyers rely on an array of toolsets to help them research and assess sites. In addition to ratings tools, old-fashioned search ranks way up there as does reading media news and solicitations by diligent media reps. Create your own media databases so you can record smaller, less popular sites you find. Keep your eye out for social ratings tools, like those found on Balihoo or Technorati. Of course, rely, too, on your own historical experience with a site and a particular kind of advertiser.

Experience counts for a lot in this business.

ClickZ Live Chicago Join the Industry's Leading eCommerce & Direct Marketing Experts in Chicago
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 4 days and 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda and register by Friday, Oct 3 to take advantage of Early Bird Rates!


Hollis Thomases

A highly driven subject matter expert with a thirst for knowledge, an unbridled sense of curiosity, and a passion to deliver unbiased, simplified information and advice so businesses can make better decisions about how to spend their dollars and resources, multiple award-winning entrepreneur Hollis Thomases (@hollisthomases) is a sole practitioner and digital ad/marketing "gatekeeper." Her 16 years working in, analyzing, and writing about the digital industry make Hollis uniquely qualified to navigate the fast-changing digital landscape. Her client experience includes such verticals as Travel/Tourism/Destination Marketing, Retail & Consumer Brands, Health & Wellness, Hi-Tech, and Higher Education. In 1998, Hollis Thomases founded her first company, Web Ad.vantage, a provider of strategic digital marketing and advertising service solutions for such companies as Nokia USA, Nature Made Vitamins, Johns Hopkins University, ENDO Pharmaceuticals, and Visit Baltimore. Hollis has been an regular expert columnist with Inc.com, and ClickZ and authored the book Twitter Marketing: An Hour a Day, published by John Wiley & Sons. Hollis also frequently speaks at industry conferences and association events.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Media newsletters delivered right to your inbox. Subscribe today!



Featured White Papers

IBM: Social Analytics - The Science Behind Social Media Marketing

IBM Social Analytics: The Science Behind Social Media Marketing
80% of internet users say they prefer to connect with brands via Facebook. 65% of social media users say they use it to learn more about brands, products and services. Learn about how to find more about customers' attitudes, preferences and buying habits from what they say on social media channels.

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising
If you're considering implementing a marketing attribution model to measure and optimize your programs, this paper is a great introduction. It also includes real-life tips from marketers who have successfully implemented attribution in their organizations.


    • Tier 1 Support Specialist
      Tier 1 Support Specialist (Agora Inc.) - BaltimoreThis position requires a highly motivated and multifaceted individual to contribute to and be...
    • Recent Grads: Customer Service Representative
      Recent Grads: Customer Service Representative (Agora Financial) - BaltimoreAgora Financial, one of the nation's largest independent publishers...
    • Managing Editor
      Managing Editor (Common Sense Publishing) - BaltimoreWE’RE HIRING: WE NEED AN AMAZING EDITOR TO POLISH WORLD-CLASS CONTENT   The Palm...