What Gmail Teaches Us About Spam Filtering

• Tweet
• Buffer

In March Google added more detailed information about why messages are being filtered as spam in Gmail. Looking at that information provides some interesting insights into what's going on, how Google is analyzing and categorizing email, and even explodes a few myths about deliverability. The value of this insight goes beyond users with an @gmail.com address. Google has combined its Postini spam filtering service into Gmail so the learnings here are applicable to the four million businesses using Google apps and enterprises using Postini directly. In addition, how Google categorizes spam is not that different to other major ISPs, so many of the learnings will apply elsewhere.

To see this information yourself simply go into your Gmail spam folder and open one of the emails. Look just below the header (the bit that shows who it's from, who it's to, and when it was sent). Here you'll find a description of why it is in the spam folder and a link to learn more.

There are two primary reasons why a message may be in the spam folder. The first is that Google considers it a potential forgery or phishing message and the second is that Google considers the message to be spam. There are multiple causes and reasons behind each of these dispositions and Google does provide some more details.

I'll start with the forgery and phishing disposition. Phishing is the act of sending an email to a user falsely claiming to be a legitimate enterprise in an attempt to scam them into surrendering private information that will be used for identity theft. Forged emails are those that are not from whom they purport to be. All phishing emails are forgeries but not all forgeries are phishes, hence the distinction in the Gmail system.

Messages that indicate this disposition include:

1. Our systems couldn't verify that this message was really sent by yyy.com.
2. This message may not have been sent by xxx@yyy.com.
3. Similar messages were used to steal people's personal information. Unless you trust the sender, don't click links or reply with personal information.

If you're a marketer and your email is being marked as a phish or forgery the problem is almost certainly technical. Avoiding being marked as a phish or forgery is just a matter of having a properly configured email infrastructure, of dotting the i's and crossing the t's. To do this takes three steps.

The first is to confirm that there are no technical errors in your messages. This should be handled by your email service provider or technology group. Your sending email servers should have valid name entries (aka forward and reverse DNS), content should be appropriately encoded, and the messages should adhere to email standards (aka RFCs). The second is to verify your from and reply-to addresses. These should be valid and owned by your organization. Also confirm that the domain you're utilizing is owned by your organization. Ideally make sure you're using your commonly recognized domain. This is especially important if yours is a well-recognized brand. Sending from email.mycompany.com is better than mycompany-email.com since the latter could be a domain registered by an unscrupulous third party.

The final step is to implement authentication. Authentication confirms to recipients (in this case Google) that the messages really were sent by you.

There are two ways to do this: Domain Keys Identified Mail (DKIM), which cryptographically signs every outbound message and so proves the message was not tampered with and was sent by who it claims. The other is Sender Policy Framework (SPF), which simply defines which email servers are permitted to send email for your organization. You should implement both technologies. Some recipients check only one but many check both of these.

Take these three simple steps and Gmail (and other ISPs) will be left in no doubt as to the authenticity of your messages, and warnings about forgeries and personal data theft will go away.

In my next column I'll look at the causes of the spam disposition, what it tells us about how Google categorizes and filters email, and what steps you can take to prevent it.

Until next time,

Derek