Between “Bounced” and “Delivered”

What do you call this equation:
(# of emails sent) – (bounces) = ?

Does it seem like a trick question? In a way it is, and in a way it isn’t. Most email marketers, ESPs and others who care (including myself) historically refer to this figure as “email delivered,” “total email delivered” or (when expressed as a percentage) “percentage of email delivered” or “delivery rate.” But I’m starting to rethink that. Just because an email doesn’t bounce doesn’t mean it’s delivered to the inbox. Most spam filters don’t let a sender know when email is blocked or redirected to the junk mail folder. And the incidence of false positives, along with concern over them, is growing:

  • In March of 2005, a Return Path study found that, on average, 22.0 percent of opt-in email messages, more than one in five, were incorrectly filtered as spam and not delivered to the inbox

  • In June of 2005, Pivotal Veracity studied 100 household-name companies over a six-week period and found that 54.0 percent of them had problems getting some of their email messages delivered to the inbox
  • In August of 2005, a Socketware survey found 68 percent of email marketers citing deliverability as their top concern .

Let’s do the math:

(5,000 bounces) / (100,000 emails sent) = 5 percent bounce rate
(100,000 emails sent) – (5,000 bounces) = 95,000 emails?

Can we really called these 95,000 emails “delivered” when, according to the Return Path study, there’s a good chance only 74,100 of them actually make it to an inbox?

(95,000 emails)/(100,000 emails sent) = 95 percent ? rate

Can we really say we have a 95 percent “delivery” rate when it’s quite possible only 74.1 percent of the emails reached their intended destination?

Yes, I know it’s semantics. But this is an important number. It’s the denominator in a number of key metrics including:

  • Open Rate (opens) / (# emails sent – bounces)

  • Click-through Rate (clicks) / (# emails sent – bounces)
  • Conversion Rate (orders) / (# emails sent – bounces)

I’m concerned terming the (# emails sent – bounces) figure “delivery rate” provides a false sense of security. When a client says “My delivery rate is 96.2 percent; I don’t have a deliverability problem,” for example.

Or when an ESP tells me, “Your delivery rate was 95.7 percent; you don’t have a deliverability problem.”

What level of confidence can we have in that figure?

There are multiple levels of spam filters out there to deal with. Once, we were frustrated we had to work with the spam filters of every individual ISP we sent to. Add today’s corporate and desktop filters and the challenge has more than tripled.

  • Do we have good relationships with the top ISPs we send to so our emails get through?

  • Have we contacted the biggest corporations we send to and requested to be whitelisted?
  • What percentage of our individual email recipients have added our “from” address to their online address book or filter whitelist to help our email reach their inboxes (and ensure our HTML images appear and links are clickable)?
  • Can we really be certain all email we send that doesn’t bounce is “delivered?”

DoubleClick, in its newest email trend report, calls this figure “non-bounces,” which it defines as (1 – the bounce rate). This is a step in the right direction, but I’m not sure it will stick. The term “non-bounces” sounds a little too trivial to represent the bulk of email we’re depending on for a response.

Some ESP tracking systems skirt the issue entirely. Two I’m familiar with report the number of emails sent, and the number and percentage of bounces. But they fail to connect the dots and present a raw number, percentage or name for emails that didn’t bounce. This leaves me to calculate (and to name) that figure myself. Not the best solution.

I’ve taken to referring to this metric (# of emails sent – bounces) as “emails assumed delivered.” It’s a mouthful, but I feel it’s a more representative title. It doesn’t sugarcoat the issue and suggest bounces are the only obstacle to getting into the inbox. It doesn’t misrepresent the likelihood of all these emails getting delivered. And it allows some margin of error.

What do you think? Has the term “emails delivered” outlived its usefulness? Should we be looking for another term to describe (# emails sent – bounces)?

Related reading