Handling Spam Complaints: Suspect Messages

In my last three columns, we discussed feedback loops, an initial analysis on complaints per e-mail address, and a second analysis on the source of the e-mail addresses that complained. Today we’ll talk about the final of three analyses I do on spam complaints for clients to identify the underlying causes and address them before they become problems.

The data I’m using is real – these articles detail an actual spam analysis that I did for a client. In this case, they were already being blacklisted before we got the data from their e-mail service provider (ESP) to analyze. In reality, you should be monitoring your feedback loops and doing these analyses on a monthly or quarterly basis, so that you can identify any potential issues before they become serious problems.

In this final analysis, we’re going to look at which e-mail messages generated the spam complaints.

Rather than look at the raw number of complaints per e-mail, you need to look at the spam complaint rate by dividing the number of complaints by the quantity of e-mail messages assumed delivered (quantity sent minus bounces). This provides you an apples-to-apples comparison that holds up even when some sends are very large and others very small.

As we discussed in a previous column, every ISP and anti-spam organization uses different guidelines to determine when to blacklist or block your e-mail messages. That said, there are some benchmarks you can use.

With my clients, I consider a 0.1 percent spam complaint rate (one complaint per thousand e-mail messages sent) to be where deliverability problems begin. Some ISPs have thresholds as high as 0.3 percent (three spam complaints per 1,000 e-mail messages sent); once you reach this level, you are very likely to have deliverability issues. These two thresholds, 0.1 percent and 0.3 percent, are represented on the bar chart below by the orange and yellow horizontal lines.

The average and median spam complaint rates are also noted with a yellow arrow; both were 0.31 percent. So on average, the sends for this period were above the high threshold for complaints.


In the bar chart above, each blue bar represents a single send. During the period analyzed, which was about three months, there were 32 unique sends, hence 32 blue bars. The first thing you’ll notice is that all but one, which is 97 percent of sends, generated at least one spam complaint. In fact, 97 percent of sends had spam complaint rates at or above 0.1 percent, the low threshold for blacklisting issues.

Looking at the higher threshold of 0.3 percent, more than half the sends (53 percent) had spam complaint rates that met or surpassed it.

Although anything above 0.1 percent is a potential issue, I’ve only highlighted spam complaint rates at or above the 0.3 percent high threshold in the following table, because otherwise all but one row would be yellow.

This table sorts the sends by the division which was responsible for the message and then by the product or campaign the message featured. Of most concern here is the large block of yellow in the middle:


Both of division E’s sends garnered a spam rate of 0.4 percent or higher. Four out of five division G’s send garnered spam complaints at or above 0.3 percent.

Also of primary concern: division K sent only one message, to a relatively small quantity of recipients, and it had a spam complaint rate of 0.57 percent, the highest of all the sends.

The first thing to probe into more deeply here is what these divisions are sending and why such a high percentage of recipients perceive these messages to be spam. We did this, and once we had some hypotheses to explain this, we began testing factors that included segmentation, targeting, content, and creative to reduce the complaint rate.

A shortcut way to try to quickly decrease the spam complaint rate would be to include an unsubscribe link right at the top of the message. I’m not generally a fan of this, because while you’ll decrease your spam complaints, you’ll still have people removing themselves from your list via unsubscribe, since the underlying issue that caused them to dislike the message hasn’t been addressed. I prefer to try to understand what’s causing recipients to perceive the e-mail as spam and address that problem head on; it’s a better strategy for keeping more people on your list and engaged in the long term.

We also took a look at the differences between campaigns sent by the same divisions which garnered very different levels of complaints. For example, campaigns from divisions A, C, D, G, I, and J. In many cases (but not all), the smaller send quantity garnered fewer spam complaints and when we investigated we found that for the smaller send, the division had actively segmented the list and targeted content, compared to larger sends by that division, which were “batch and blast.”

So the key takeaway from these four columns is that you should monitor and analyze your spam complaints on a regular basis, preferably before they surpass the 0.1 percent threshold. By identifying and addressing the issues causing people to report your e-mail as spam, you can not only decrease your spam complaint rate and risk of being blacklisted, you can improve your long-term relationship with those on your list.

If your organization isn’t doing this currently, this month is the time to start.

Until next time,

Related reading

Flat business devices communication with cloud services isolated on the light blue background.