The Paradox of Measurement
As an industry, we have positioned online to be the most measureable of all media. But have we picked the wrong metrics?
The rise of the quant on Madison Avenue has led to many analogies that bridge the financial services industry with that of digital marketing.
High frequency trading = real-time bidding (RTB)
Spot markets = ad exchanges
Futures markets = insertion orders with option exercise fees
Commodities market = blind inventory
(Please insert your favorites in the comments section.)
I have used many of them and find them both helpful and fun. However, there’s one big analogy that’s missing from the quiver – what is the marketing equivalent of currency? I don’t believe there’s an appropriate answer.
What I mean by currency is a cold, hard measure of objective value. Without it, all transactions are relative.
The primary problem is not that a currency does not exist but that while a currency has been promised, false positive or relative metrics have been offered in its place. This is because, as an industry, we have positioned online to be the most measureable of all media. I believe we have picked the wrong metrics.
Consider CPA. It seems to be a clear marketing objective, but on closer inspection, I would contest that it’s often more misleading than not.
Example 1: Cookie Bombing
At one point in my career I had the privilege to work with a digital marketer that tried to spend $20 million in one month in digital display. During that time period, the digital channel could take at least partial credit for almost every online conversion. The client also realized eCPMs that were near $0.10, so the effective reach and frequency of the campaign was absolutely massive.
This is the same tactic to a lesser degree that is currently deployed by many affiliate networks and current sellers of CPA. It is commonly known as cookie bombing and it works if you’re using DFA to measure effectiveness, but it does nothing to strengthen brand marketing effectiveness. The tell-tale sign of a cookie bombing marketing plan is when nearly 100 percent of all conversions are claimed by the digital marketing plan and there is no correlation between whether a campaign/partner serves zero or 50 million ads in a day, and actual real activities generated that day or at lag = t-1, lag = t-2, lag= t-3 (time lag will capture any latent effects of the advertising).
Example 2: Below the Fold vs. Above the Fold
My company’s research suggests that pages below the fold are two times more likely to receive attribution credit than pages above the fold.
| Impressions | Conversions | |
| Ratio BTF:ATF | 5.4 to 1.0 | 10.7 to 1.0 |
| Probability | 1 | 2 |
| *sample 60,000,000 Purchased Impressions | ||
If advertisers are measuring solely on last view-through attribution, the RTB algorithm of their demand-side platform will gravitate to either one of two options:
- Domains that only serve below-the-fold placements
- If their algorithm uses ATF/BTF as an attribute, the algorithm will actually pay a premium to place an ad below the fold
This again is gaming a flawed measurement criteria instead of fulfilling the advertiser’s desire to have their ads seen in well-lit, uncluttered environments.
I believe this to be true because the last ad on a page is the last ad to load so it will receive the last impression attribution credit.
Example 3: Upper-Funnel vs. Lower-Funnel Tactics
Today, most optimization methods gravitate toward the use of retargeting. This supports the buyer’s request of optimizing toward attributed credit or view-through optimization.
Retargeting at its best is a friendly reminder to a likely purchaser to come back to the advertiser’s site and purchase. This is the goal.
In reality, retargeting often takes credit for shoppers that are likely to purchase a product anyway. Even when done right, retargeting takes more credit than it is due.
In a case study done by my company for a U.S. airline, we measured the extent that our own retargeting line items “stole” attribution credit from upper-funnel-focused tactics. We did this by comparing the activity allocation of the upper-funnel tactic and the retargeting tactic through a first-impression vs. a last-impression attribution model.
The results of the first-impression attribution model showed that our own retargeting impressions diverted approximately 40 percent credit from the upper-funnel tactics. If the advertiser’s agency canceled the upper-funnel line, the advertiser’s CPA would go down; however, impression volume would suffer over the long term. Unfortunately, despite our sincerest objections, this is exactly what the agency for the advertiser decided to do.
Conclusion
A single publisher, ad network, ad server, DMP, or DSP that hides their marketing tactics and algorithmic approach as part of their “secret sauce” cannot fix the game. It’s the assembly of all the parts that delivers the full value.
It can only be done through a targeting platform that ties together multiple targeting tactics, algorithms, and any inventory (RTB or publisher direct) under a single arbitrator of value.
Learning and tactic transparency are critical in order to train the media planner and advertiser on the relative value of each tactic for their brand. It trains their marketing intuition. But this approach also disrupts the current state of media planning and therefore full adoption via platform consolidation is a necessary but difficult leap of faith.
It’s further complicated by the fact that even though there are clear benefits to the advertiser, platform consolidation puts the agency model at risk. This is similar to what we’ve witnessed over the past two year as the ad exchanges/SSP/DSPs have put the ad network model at risk. However, the networks that provide value through meaningful and tangible differentiators have done far better while those that have not have done far worse. The same should be true for digital agencies as we see more targeting platform consolidation over the coming years.
My problem with CPA is not that it’s used as a measurement but rather that it’s used as currency to pass judgment on the effectiveness of one “inventory” supplier over another when that measurement is clearly flawed. Bad actors will always game univariate measurement. It’s in their best interest to do so.
One of the most heralded benefits of digital – its ability to measure – has slowly become a major deterrent to good advertising. Instead of informing strategy, it has begun to distract it; instead of pointing our efforts in the right direction, it has veered them off course.
But there is much to be learned from the game, as well. Until planning teams become more educated about these phenomena, the industry should view CPA metrics with a cautious eye. This doesn’t mean dismissing them altogether, but simply recalibrating them with the understanding that CPA is a relative, not objective measure.
As long as digital marketers continue to offer the current state of CPA as currency, then marketers will continue to underinvest in digital. Digital media was promised as the most measurable media, but we got the measurement wrong. Our mistake keeps scale out of the industry. It’s a cruel paradox, but one I believe to be true.
