More NewsSpiders and Robots Are Ghouls and Goblins

Spiders and Robots Are Ghouls and Goblins

After hashing out a media glossary last week, IAB takes aim at spiders and robots by tapping ABCi to create a master list.

In a move to improve online ad campaign measurement, auditing specialist ABCi Monday pledged to create and maintain a master list of spiders and robots, which are widely acknowledged as the main culprit for inaccurate measurements, for the Interactive Advertising Bureau (IAB).

Why is a list important? Spiders and robots automatically search Web content for information and are used to: retrieve data about content for search engines, gather content pages for personal offline viewing and track advertising activity by Web site and by advertiser and identification of “What’s new information”. Because there may be multiple robots or spiders on a site, sites may have a problem distinguishing between non-human spider and robot activity and consumer interactions, which throws off the metrics reported to advertisers.

Robin Webster, president and chief executive officer of the IAB, extolled the virtues his organization’s plan, claiming the robot/spider list will cut back “inconsistent counts.”

“The primary benefits of the list will be to reduce the time, effort and cost to publishers, third party ad servers and agencies associated with report reconciliation, and to provide advertisers with accurate measurement metrics,” Webster said. “By utilizing this approved and constantly updated master list, publishers will be able to filter out robot traffic from their reporting to advertisers, instilling greater confidence in the medium.”

The IAB is in the midst of a multi-step process to hash out ad measurement protocol. The firm established a media glossary last week when it was determined that third-party ad servers and publishers defined such terms as ad impressions, unique users, clicks, total visits and page impressions, disparately.

Once the robot/spider list is in place, IAB said the next step will be for ABCi to examine the effect of spiders and robots on the two different methods of ad serving: client-initiated and server-initiated, followed by the release of measurement guidelines by year-end.

Having maintained a robot list compile from over 8,000 audits and 200 trillion transactions since May 1995, it is logical that ABCi is qualified to compile the list and conduct further examinations of the ad measurement process. ABCi invites other organizations to suggest updates to the list, so long as proper documentation is supplied. On the other hand, ABCi also welcomes challenges to the list, also accompanied by proper documentation.

Available for free to IAB members and ABCi clients, the robot/spider list will be updated monthly, and serves as the third step in a slew of recommendations the IAB will post by year’s end.

Related Articles

GDPR: The role of technology in data compliance

Data & Analytics GDPR: The role of technology in data compliance

3w Clark Boyd
What companies can learn from the We-Vibe lawsuit about the Internet of Things

Legal & Regulatory What companies can learn from the We-Vibe lawsuit about the Internet of Things

8m Al Roberts
Has advertising arrived on Google Home?

Media Has advertising arrived on Google Home?

8m Al Roberts
Is Twitter slowly dying?

More News Is Twitter slowly dying?

9m Al Roberts
FedEx launches fulfillment service to take on Amazon

Ecommerce FedEx launches fulfillment service to take on Amazon

9m Al Roberts
Target is the top retail digital marketer, so why is it struggling?

Ecommerce Target is the top retail digital marketer, so why is it struggling?

8m Al Roberts
YouTube is "on pace to eclipse TV" thanks to savvy algorithm use

More News YouTube is "on pace to eclipse TV" thanks to savvy algorithm use

9m Al Roberts
YouTube is getting rid of 30-second unskippable pre-roll ads

Ad Industry Metrics YouTube is getting rid of 30-second unskippable pre-roll ads

9m Al Roberts