Click-to-Open Ratio and Its Shortcomings

Recently I’ve found myself discussing a commonly used e-mail marketing metric that, quite frankly, has me puzzled. The metric in question is “click-to-open” (CTO) or “click-to-open ratio” (CTOR) and a number of marketers use it. I’ve seen it described as “an important measure of the effectiveness of your e-mail message.” One e-mail service provider’s glossary apparently shares my puzzlement stating, “The Click-to-Open-Ratio is supposed to be a significant indicator of the quality of your campaign.” What has me confused is: what precisely it is used for and why?

CTOR is a ratio or ratios, a second order metric. In the Email Experience Council’s standardized e-mail marketing terms, CTOR is defined as clicks divided by e-mail renders. Most people call this click-through rate divided by open rate. The click-through rate (CTR) is the number of times a message is clicked divided by the number of accepted messages (traditionally called deliveries). The render rate is the number of times a tracking image is downloaded divided by the number of accepted messages.

Though critical for CTOR, since it is the divisor, render rate is problematic. Depending on whom you talk with, they may be discussing the unique render rate (which is what is recommended by the EEC), total render rate, or even confirmed open rate. Confirmed open rate being unique render rate plus any users who clicked but did not register an open.

Therein lies the first of my concerns with CTOR. When marketers disagree on the calculation of a metric, the metric’s value significantly declines. Open rate is such a metric. Open rate values should always be approached with caution due to high variability. My concern is that when CTOR is used instead of CTR (and every client I know of that uses CTOR in practice uses it in preference to CTR) the lack of consistency of open-rate calculations is spread into the click-through figures too.

While reading up on why CTOR is useful, I found claims that non-openers can’t click and when considering clicks we shouldn’t care about those who can’t click. I don’t find this logic compelling. In most cases non-openers can click. The same piece argued that using CTOR evens out the variances in CTR. Given that both can be considered positive responses to a mailing, it’s hardly surprising that if you divide one by the other and end up with a relatively even proportion. Therein lies my second concern. You have to be very careful with second order metrics as they can introduce unexpected variances and the logical relationships become complex.

CTR varies primarily based on the responsiveness of recipients receiving a mailing. Clearly there can be many reasons that such responsiveness varies including whether the mailing made it to the inbox, how the creative was presented, their engagement level, and more. However there is an additional set of reasons why open rate may vary including sender whitelist status, e-mail client platform (mobile, desktop, Web mail etc.) in addition to recipient responsiveness. The result is that open rate and CTR vary for overlapping but often independent reasons. This introduces far more complexity when comparing CTOR rates from mailing to mailing and particularly from campaign to campaign. Similar CTOR rates may mask widely disparate response rates and return on investment.

These two issues lead to my final concern. What does CTO tell me that CTR does not? I have yet to hear a cogent explanation of this. My cynical side suspects that since the divisor for CTOR is smaller than that for CTR it’s being adopted by some because it’s like CTR but bigger. If that’s the case, people are being misled because the assumption isn’t true. Yes it’s bigger, but it’s not just like CTR.

The key takeaway? As a marketer, you must clearly understand what metrics you’re using, why you’re using them, and what they tell you. This requires an understanding of both the mathematics (statistics) and how those metrics reflect the behaviors and activities you wish to monitor.

Having come out against click-to-open, I’m very interested to hear from anyone who can enlighten me as to what CTOR can show me that click-through rate does not.

This column was originally published on Sept. 2, 2010 on ClickZ.

Related reading