Over the next few weeks, I’m going to get back to the basics of Media 101 and address the sizeable topic of dealing with performance data. I’ll start by describing the types of measurement data.
Levels of Site Performance Data
Your campaign’s objectives dictate which types of measurements you’ll use to evaluate the performance of your media buys. I spoke of the different types of objectives and the corresponding metrics in a previous column. These data types fall into three major categories:
- Tier I: Basic impression data (e.g., numbers of impressions delivered)
- Tier II: Simple performance measurement data (e.g., numbers of sales generated)
- Tier III: Individually tracked performance measurement data (e.g., sales generated by individual viewers of an ad)
Each data type warrants a different mechanism of data collection and involves different types of analyses at the end of a campaign.
Before examining data collection and some practical steps to conducting campaign performance analysis, I’d like to further define each data type.
Tier I: Basic Impression Data. Despite our ambitions to be more clever about our buys and to exploit more complex data, about 99 percent of media buys are still conducted with simple CPM deals. These relatively plain transactions use basic impression data to measure compliance. So many thousand impressions are sold, and at the end of a campaign the impression levels are checked to verify completion.
Tier II: Simple Performance Measurement Data. Even with simple CPM deals, which require impression data to ensure compliance, advertisers frequently desire additional information to better determine which ads and which placements have the most effect on performance measures, such as click-throughs and sales. Buyers will frequently want to see data such as the following:
- The number of click-throughs generated
- The number of transactions inspired by a placement
- The number of submissions or entries resulting from a placement
This information can then be used to judge individual pieces of creative or the individual Web sites carrying the ads. Frequently, a site that seems efficient in terms of cost per impression will be less efficient in terms of other measures, such as cost per transaction, or cost per dollar of profit generated.
Importantly, these “simple” performance measurements are limited to gross numbers of certain performance metrics. You are not measuring the behavior of individuals; instead, you are looking at the gross number of certain types of transactions, like clicks or purchases.
Tier III: Individually Tracked Performance Measurement Data. You get more personalized with the next level of data because you begin to track performance measurements against real people. This becomes important when you want to see if individuals become more likely or less likely to purchase products sometime after they see an ad. Following a customer purchase, some companies can go back to their database to see which ads that individual has seen over the course of a recent period of time. This helps the advertiser figure out which ads are producing the desired effects, however latent. This becomes a fairly important factor in certain product categories, like those that involve very high-cost items or other goods that require a great deal of consideration and comparison.
This third level of measurement data also comes into play when trying to determine which types of ads or media placements are most effective for different types of people. By collecting information about these individuals, a company can help narrow down the targeting and creative that will make those individuals most susceptible to the company’s messages.
Of course, keeping track of this data becomes very involved. You need to keep unwieldy databases, get permission to employ cookie information from the Web sites you purchase from, and set up data-tracking mechanisms on the client’s site.
Putting these systems in place, managing them, and interpreting the results is an expensive, time-consuming process. Because it involves the client’s site and the unique types of information that the client desires, there aren’t off-the-shelf products to conduct this type of measurement. Each company conducting tier III data analysis creates its own custom solution — one that requires a great deal of rewriting whenever a site is redesigned or otherwise modified.
Before addressing these gory details, however, you should make sure your campaigns do not run afoul of privacy standards (to be covered next week).
Is It Worth Setting up Tier III Data Collection?
Collecting all the data necessary to track the people who are seeing the ads and their later purchases is expensive. Just banner serving will add another half-dollar to the CPM, and tracking these individuals involves serving pixels at every page between them and their subsequent transaction. With CPM rates frequently dipping below $5, it’s not uncommon for the tracking expense to exceed the media cost.
But the media cost is seldom the main reason why agencies and clients fail to collect all the data necessary for tier III analysis. The primary reason is that it’s just terribly difficult to collect all the necessary data in such a manner that an analyst can link one datum from one source to other data from other sources.
It’s relatively simple to be able to say a company received 1,000 impressions, 10 click-throughs, and 2 sales. It’s extremely difficult to say that one of those sales came from an individual who saw a particular ad in a particular place — and even more difficult to be able to say that the other individual is of the same demographic yet saw a different ad in a different place.
All of the factors have to be controlled very carefully. If any one of the factors in this measurement scheme becomes divorced from an individual’s file in the database, then the rest of the data becomes useless for tier III analysis.
A common disconnect occurs when a client’s IT department doesn’t make the marketing group’s requests for data a high priority. IT groups never had to deal with vendors like ad agencies before, and, frequently, there’s a great deal of indifference, if not friction, between the two.
A company’s internal IT department may not be open to the idea, or perhaps may not be nimble enough, to provide the customer data from the company Web site in the right format. If that happens, then all the cookie information and other efforts conducted with the ad selling Web sites are rendered useless. If the client can give only aggregated sales data, or data on individuals who cannot be directly linked to the cookies, then the data becomes too dirty and no useful conclusions can be drawn about creative or media choices.
Keeping the data clean enough to draw useful conclusions is like trying to keep a large house warm during a New Hampshire winter. You need to have every door and every window closed. With just one open, you might as well save money and turn off the heat. If, for instance, you can’t get a Web site to send users seeing different pieces of creative to different client URLs, it might not be worth the cost of setting up data-tracking mechanisms to track individuals from certain ads and media buys.