Measuring What Matters

One of my favorite scenes in the “Star Wars” trilogy comes from “The Empire Strikes Back.” Luke Skywalker runs into a difficult challenge while training under the Jedi master Yoda, and he ultimately fails to use the Force to his advantage. Yoda’s advice to Luke was somewhat cryptic, but sound. “Unlearn,” said Yoda. “You must unlearn what you have learned.”

Luke was having trouble using the Force to lift his X-Wing Fighter out of the swamp in which he had crash-landed. His preconceived notions concerning the heavy weight of the spacecraft were what ultimately caused him to fail. In telling Luke to “unlearn,” Yoda was giving Luke some good advice: Don’t let your past experiences and preconceptions hold you back.

Sometimes I feel as if the online advertising industry needs Yoda more than it needs traditional-advertising veterans who try to make Internet-advertising success measurement fit into preconceived models that are better suited to traditional media. While I do feel that Internet advertising needs to take its rightful place next to television, print, radio, and other media as a valuable component of a marketer’s media mix, I don’t think it’s wise to try to force its measurement into a model that is many years older than the Internet itself.

Early Online Measurement

In the early days of Internet advertising, we often gauged the success of an online campaign by measuring click-through rate (CTR). At the time, this made sense, as online marketers tried to draw a parallel between CTR and “open rate,” a measurement that is used often in traditional direct mail campaigns. What we didn’t understand is that there is a fundamental difference between online ads and traditional direct mail.

For one thing, banner ads cannot be thrown onto a coffee table and opened later at the prospect’s convenience (unless every Internet user were to take advantage of a service like AdClip, but I digress). We began to see that CTR could measure only a prospect’s immediate interest in a product or service, noting its weakness as a success metric.

Since that realization, we have heard several different methods proposed as the successor to CTR and have been somewhat successful at weaning advertisers off CTR as the ultimate determination of success. Some media buyers advocate using online surveys to measure success. Others want to use tools to gauge the makeup of a site’s audience and make audience-based assumptions about who they’re reaching with their advertising. Yet others want to employ reach and frequency metrics to their online advertising and treat online advertising like just another medium in their mix.

When I planned and bought traditional media early in my career, I remember being given a reach curve for network radio by my supervisor. It looked like a 20th-generation photocopy that had been originally taken from an advertising textbook many years earlier.

The curve allowed a planner to make some assumptions about reach and frequency based on the number of rating points purchased and the number of radio networks used to reach the audience. Using this method didn’t take into consideration several factors that many would consider to be integral to the success of the ad campaign: the editorial environment in which the ads were placed, the loyalty of an audience to a particular radio show, the response to the ads, and the effect of those ads on the mindset of the audience. At the time, it struck me as a very strange and inadequate way to measure advertising.

What Measures Success?

While I’m eager to see online advertising take on a role in every traditional advertiser’s media mix, I’m not sure that the tradeoff of ignoring this medium’s unique characteristics is worth it. I think every online ad campaign needs to address the fundamental reasons why an advertiser is advertising in the first place in order to be deemed successful or unsuccessful.

Advertisers advertise because they want a transaction to take place. That transaction can take many forms, depending on the advertiser’s strategy. It could be the direct purchase of a product or service. It could be the prospect’s commitment to a charity or a cause. It could even be something as subtle as a “transaction of the mind,” in which a prospect takes into consideration a particular product or service.

These transactions are what advertisers want. We owe it to the medium to use technology to directly measure the volume and effect of these transactions as they relate to the ad campaigns we place. If advocates of print, radio, and television advertising could measure these transactions directly in a similar fashion as online advertising can be measured, they would. (And some developments in interactive television may yet allow TV to do this in the future.)

I think we could all stand to “unlearn” some of the ways we measure traditional advertising in order to avoid failing to take full advantage of the way technology allows us to measure online advertising. We cannot afford to make another mistake like the one we made when we decided to let CTR be the most popular measure of online success.

Instead, we need to make use of online surveys, clickstream data, log files, third-party ad servers, and behavioral data to measure more closely the things that matter to advertisers — transactions. If we adopt an audience-based or reach/frequency model now, we won’t be able to take full advantage of the trackability of online media, which would be a shame.

While no online-measurement company is yet capable of isolating and measuring the full effects of online advertising, we’ve seen plenty of innovation in this sector, and we need to continue monitoring the latest developments in measurement in order to allow advertisers to truly know what their online advertising is achieving.

Related reading

prog
hillary-clinton-text-message-signup
screen-shot-2016-09-13-at-10-20-04
mike-andrews
<