Romney Ran on Other Gay Sites: Story Prompts Major Nielsen Data Revision

Last month, The New York Times reported the campaign for Republican presidential primary contender Mitt Romney ran thousands of ad impressions on Gay.com in August. It turns out, apparently a result of ad network targeting or lack thereof, the campaign also ran about 5,000 display ads on other gay-centric sites Advocate.com and PlanetOut. That’s according to Nielsen Online AdRelevance.

When I first read the Times piece, I was puzzled. After all, I’d sifted through AdRelevance data for the same period as part of ClickZ’s ongoing Campaign ’08 coverage back in September. Had I overlooked this juicy detail about Romney’s ads when AdRelevance sent me the August data?

Nope. After contacting Nielsen Online, I learned that the August information on political advertisers they’d originally sent me, which did not list Romney as an advertiser, had since been revised to include the Romney ads. Those updates were prompted by inquiries for The New York Times story, they said.

But it didn’t end there. Incomplete data from AdRelevance had found its way into six stories published by ClickZ between April and September of this year. What follows is a detailed description of how I became aware of the data discrepancies.

Obama Ads Uncovered Along with Other Discrepancies

According to the original data, the only presidential campaign in the “political advertisers” category in August was John McCain 2008. However, the new data showed in addition to the uncovered Romney ads, Democratic Senator Barack Obama’s campaign also ran display ads that month.

These revelations were disconcerting for me as a reporter. Not only have I received AdRelevance data on political campaigns for years now, I’ve developed a new ClickZ Campaign ’08 News sub-section devoted to covering digital ad and marketing efforts by the 2008 presidential campaigns.

At ClickZ News, we pride ourselves on getting our reporting right, but when it comes to tracking online ads in any cohesive manner, we’re only as good as the data we’re provided. Political campaigns, particularly national ones, typically are very reluctant to share any information about their tactics, including their online ad buys. And Nielsen Online is the only measurement firm out there that tracks online display ads run by political advertisers in any regular, quantifiable manner that will supply this information to ClickZ.

Keep in mind, I realize information provided by Nielsen and others is not always 100 percent accurate, and when I’ve been aware of data gaps, I’ve made note of it. For instance, when covering July presidential campaign ads, I wrote that although AdRelevance did not report them, “PointRoll expandable video banners for Senator Barack Obama have been spotted around the Web this month, including on UnionLeader.com, where early primary voters in New Hampshire reading the Manchester-based newspaper site would be sure to catch them.”

The Flood Gates Open

And what about previous months? I’d been reporting on AdRelevance data on the presidential campaigns since April. It didn’t take long to learn the research firm had revised its data on political advertisers for previous months, too, but ClickZ had never been notified. It seems as though the errors would have lived in perpetuity on ClickZ had I not investigated the August Romney data discrepancies.

ClickZ recently was provided with updates of all revised data we originally reported between April and September; changes in the process for compiling and reporting on political ads were reflected in data provided to ClickZ for October. After receiving updates, I found the original information was off by a little or a lot each month, in some cases missing millions of ad impressions or skipping campaigns for presidential candidates entirely, such as Barack Obama, John Edwards, and Romney.

“These are the types of feedback that we value… to continually evolve our service,” said Jason Lee, senior project manager for AdRelevance in reference to a discussion I had with him a few weeks ago about the situation.

Lee and his colleague, Jon Gibs, VP of media analytics for Nielsen Online, helped me understand what went wrong and why. “I think it has to do with the protocols we have set up for categorizing advertisers in general,” said Lee. “In the case of political advertisers, those protocols and rules may not have been flexible enough.”

In a nutshell, the AdRelevance system determines the sites it will monitor based on weekly traffic numbers. After intercepting ads running on those sites, the ads are categorized by a classification team. These are the folks who decide an ad run by John McCain 2008 or Obama for America goes in the “political advertisers” category as opposed to the auto or CPG category.

In the case of the Obama campaign ads missed in initial reports for August and other months, Lee explained, “A human did likely see the ad…. For some reason or another it slipped through the cracks.”

In the cases of ads that were ignored the first time around, most likely what happened is they simply weren’t classified, said Gibs. “We set up what we think is a very good system to track major advertisers,” he stressed, noting AdRelevance was originally designed to track big commercial advertisers — the Fords and AT&Ts of the world. “Because political flighting was very small in the beginning, it wasn’t fully on our radar screen,” he added.

“During this particular campaign season there’s a lot more scrutiny, and a lot more attention being paid to what the campaigns are doing online,” said Lee. Note, I’ve been receiving and reporting on political campaign ad data from AdRelevance for years now.

“Going forward we’re confident the data should be where it is now if not at a higher point” in terms of quality, Lee told me.

The Politics of Data

Political advertising is a beast unto itself. Understanding the difference, for instance, between Congressman Ron Paul’s campaign running ads and the Americans United for Freedom PAC running ads on his behalf will come about “by process of experience” for AdRelevance’s classification team, Lee said.

The reality is, in politics as in just about anything else, polling and other forms of measurement are prone to error. In the end, the best ClickZ can do is update our own reporting with the correct information, as well as do everything possible to vet that information. Because we can’t go back in time and see whether an ad ran on a particular site, nor can we here in NYC be served ads targeted to people living in Des Moines or people with different demographic characteristics or browsing histories, we are limited in our ability to ensure data are flawless.

Still, as online political marketing becomes more widespread, it’s important for its evolution to be chronicled. That’s my goal with previous political ad coverage in ClickZ News and our Campaign ’08 coverage. So, even when there are bumps along the campaign reporting trail, we believe it makes more sense to continue reporting these data even if sometimes they’re not perfect.

We also look forward to working with Nielsen Online and other measurement firms to continue improving the information available on online political campaigns. It’s a learning process for everyone, most of all the political campaigns themselves.

Lee affirmed this, noting, “Online advertising is obviously constantly evolving. It’s our goal is to consistently evolve our product to keep in step with that.”

For details on revised AdRelevance data for the months of April through September, check out the original stories, which have all been updated:
April 2007
May 2007
June 2007
July 2007
August 2007
September 2007