ComScore Admits Measurement Flaws

ComScore Media Metrix said on Monday it had discovered flaws in its measurement methodology, causing it to revise its site counts for October through December 2002.

The company said the methodology it introduced in October undercounted at-work users, with the restated figures causing some sites’ traffic figures and duration-of-visit data to increase sharply. Also, ComScore altered its counting of some ad-supported software, such as AWS Convergence Technology’s WeatherBug.

“It is forever a challenge in the research field to penetrate the workplace as much as one would like to do,” Dan Hess, vice president of ComScore Media Metrix.

Reston, Va.-based ComScore said its projection methodology did not capture a full representation of at-work users. Because of this, many sites had lower traffic numbers in Media Metrix’s original reports.

For example, Primedia’s network of sites, which includes About.com, saw its audience figures for December increase 25 percent to 44 million. Ebay’s audience rose 19 percent to 54 million. And Terra Lycos got a 21 percent bump to 51.6 million.

The methodology problems arose from ComScore’s June 2002 acquisition of the Media Metrix audience panel. The company acquired the panel during the unwinding of Jupiter Media Metrix. ComScore’s acquisition, however, did not include the meter software, which was sold to Nielsen//NetRatings to settle a patent lawsuit. (Jupiter Media Metrix’s research arm, Jupiter Research, is now owned by the parent company of this site.)

Without the tracking software, ComScore incorporated its own server-based measuring system with the November 2002 release of Media Metrix 2.0.

In addition to the 50,000 home Internet users recruited via Random Digital Dial sampling methodology, Media Metrix 2.0 expanded its measurement panel to 120,000, including 35,000 samples of at-work and university users.

The company started looking into its numbers when it received reports from some clients that its site statistics were off. This was nothing new, as Web sites have griped about Web measurement services under-counting site visitors since the Internet’s inception.

Hess said when ComScore re-examined its methodology, it found flaws in its at-work sampling.

“We communicated these changes to our clients, got their feedback on these changes, and overwhelmingly the response has been positive,” said Hess.

While site usage statistics do not carry the same weight as they did during the Internet boom, when companies trumpeted Media Metrix figures in their public offering filings, the vast differences between the site statistics from different measurement companies has not helped Internet publishers gain credibility for the medium.

The need for a set methodology for Web site measurement has been bandied about for some time. However, last December, the Audit Bureau of Circulations decided to fold its standalone Web site-auditing service, ABCi, into its offline offering. The auditing service is now available only to ABC members, likely decreasing the number of audits it performs.

“There’s really a need for a common set of conventions and some accountability in the form of consistent application of those conventions,” said Michael Zimbalist, executive director of the Online Publishers Association (OPA).

Hess said the restatements do not affect ComScore’s collaboration on research studies with groups such as the OPA.

Media Metrix and its rival, NetRatings, usually differ greatly in their Web site reports, leading some companies to pick and choose which figures to highlight. WeatherBug, for example, used the Media Metrix figures to label itself the Internet’s top weather site, despite Weather.com garnering more visitors in NetRatings’ rankings. Media Metrix’s re-jiggered results downgraded WeatherBug’s audience, putting it behind Weather.com.

The increase in many sites’ at-work audience could give more credence to Web publishers’ contention that the Internet is the prime media vehicle for daytime hours.

The OPA has pushed the idea of “prime time” daytime hours on the Internet. Web publishers have also begun to use dayparts, in order to charge a premium to advertisers to reach consumers during the day.