Is Anything Really Measurable Online?

So everything online is measurable, is it? Perhaps. But online is measurable in so many ways, and with so many discrepancies and inconsistencies, you almost wonder if it’s worthwhile to measure at all — at least, until some order is brought to the situation. There’s no small degree of disagreement on how measurement should be approached, let alone executed.

If the above challenges your notions of Web measurement, you didn’t attend the Advertising Research Foundation’s (ARF’s) annual audience measurement conference in New York this week.

Audits and Panels

Take the audit process that the two biggest audience measurement research firms, comScore Networks and Nielsen//NetRatings, are currently undergoing at the behest of the Interactive Advertising Bureau (IAB). IAB members are claiming massive discrepancies between their own Web stats and the panel-based numbers provided by comScore and Nielsen panels — up to two- or three-fold. This is “far beyond any legitimate margin of sampling error,” says the IAB, and not without reason. That body is questioning the wisdom of panel-based data in Web measurement.

While both Nielsen and comScore have agreed to the audits, they’re also vigorously defending their panel-based measurement methodologies. Nielsen’s Mainak Mazumdar was quick to point out server-side metrics count visits from bots, spiders, and other “machine-driven” traffic.

Then, there’s search. “I don’t know of any other way to do it than with a panel if up to 90 percent of the conversions occur offline,” said comScore chairman and cofounder Gian Fulgoni. “You can argue the brand-building value of search is being undercounted if you’re only counting clicks.”

But rapid developments in Web technology are the primary reason the panel-based companies defend their approach. “The Internet is changing so fast that if to get accredited you have to lock in one approach, you won’t be able to measure the next thing without being unaccredited,” Fulgoni claimed. Backing him up on this notion was Yahoo VP of advertising and sales insights Lynn Bolger. “We’re standing in quicksand here,” she said of new developments such as AJAX (define).

The Unmeasurable

“Standards are changing every day, and audits will become meaningless when new technologies come,” chimed in Beth Uyenco, global research director at Microsoft Digital Advertising Solutions.

High on her wish list for measurability are interactive games. “We need basic game usage statistics. We need profiles by game title. We need audience estimates by system and by title. We need to figure out a way to get good campaign effectiveness measures.”

She then reeled off a list of game platforms, including mobile, consoles (there are an estimated 7.2 million console households), instant messaging, online, MSN Games, and Windows Live. Top that off with in-game ads served dynamically (in Microsoft’s case, through Massive) and static ads embedded in the games themselves. All these platforms and ad forms are touching the exceptionally hard to reach young male demographic.

Just yesterday, in fact, Parks Associates pegged the game advertising market to exceed $2 billion in spending in 2012, making it the fastest-growing channel of any media form, including online.

Only it’s all but unmeasurable (Nielsen plans to introduce game play metrics this summer).

Other interactive technologies cited by panelists as posing the most significant measurement challenges include mobile and pod- and videocasts.

Who, and What, Is Influential?

The event took a break from debating measurability when Columbia University professor Duncan Watts appeared to challenge the notion of the influential in the marketing mix. Challenging what’s nearly accepted wisdom, Watts claims, “Overall, influentials are modestly effective. The critical people to pay attention to aren’t a small number of influential people, but the large number of easily influenced people, the ordinary people influencing ordinary people. We need to start paying attention to these networks and focus more on the other end of the influence relationship.”

So much for relying on influencers to spread your marketing message? On the surface, that appears to be what Watts is saying. But not so fast.

“Influentials cannot influence a significant number of people other than through a network,” he asserts (my emphasis). “If a network permits spread, anyone can start something. If not, no one can.”

At the risk of oversimplifying a scholarly argument, it can almost be said Watts’ argument fully supports the accepted notion of the influential — provided that influencer is operating in a highly networked interactive environment.

And that bring us back to the original question: how will we measure that audience and that sphere of influence in the future?

And can we?

Meet Rebecca at ClickZ Specifics: Online Video Advertising on July 19 in New York City.

Related reading

Overhead view of a row of four business people interviewing a young male applicant.