Email Testing: Here Come the Bridesmaids

Testing should be an integral part of any email marketing campaign. You know it, I know it, and your competitors know it. If you doubt testing can improve your results, or if you aren’t sure how to go about it, today’s column is for you.

Dessy is a major retailer of bridesmaid dresses. The company was happy with its healthy Web site traffic, but return visitor traffic left something to be desired. So Dessy teamed up with Portent Interactive, a Web marketing and design company.

The first step was to revamp the Web site. Then, Dessy added a news email feature (double opt-in, of course). Visitors can sign up on the Web site for the mailings, which feature some aspect of the site or focus on special offers.

Here’s where the testing comes in. The subscriber base was split into three groups. The first two groups were relatively small, each only 10 to 15 percent of the total database. Groupings were randomly generated.

Dessy and Portent Interactive sent one version of the message to the first small group, which we’ll call Group A, then looked at the results. They sent a similar mailing to the second small group, which we’ll call Group B. Results were compared. The better-performing message went to the remaining, large group of recipients.

Test No. 1

Dessy recently promoted The ColorFinder Manhattan store, which allows Manhattan residents to buy dresses online. You can see the mailing here.

About 1,000 members in Group A received a message with the subject line: “Live in Manhattan? Buy your bridesmaid dresses online!” It received about 300 views, and 200 people clicked through to the site.

A similar message was sent to the same number of people in Group B with one change. The email itself was the same, but the subject line read: “Manhattan Residents: Buy your bridesmaid dresses online!”

Before you read on, take a guess as to which mailing performed better. No peeking below! OK, ready?

And the winner was clearly… Group A. In Group B, the mailing generated only about 150 views and 50 click-throughs to the site. Ian Lurie of Portent Interactive and I discussed the possibilities of why Group A was more responsive, but we didn’t come up with anything definitive. My hunch: I’d rather be asked a question, as in Group A’s subject line, than be told what to do, which is the feeling I got from Group B’s mailing.

When the mailing was sent to the entire database with Group A’s subject line, it generated over 500 click-throughs. Some 400 visitors explored the ColorFinder.

Test No. 2

Here’s another test Dessy conducted. The company was promoting its LookBook, a glossy publication featuring current styles and includes a complete set of swatch cards and other tools. Portent assembled an HTML mailing that included a photo.

In early January, the mailing went out to about 800 people. The graphic was relatively small, the idea being users don’t want to wait for large graphics to download. It generated about 270 opens. Fewer than 50 recipients were intrigued enough to click through to the site.

A week later, a similar mailing was sent to another 800 people. The notable difference was a larger graphic. Guess what? The bigger image seemed to be the key. About 300 people opened the message. About 100 recipients clicked through to the site — more than double the number of the first mailing’s recipients!

Not surprisingly, Dessy sent the second mailing to the entire database, slightly less than 10,000 recipients. About 2,800 opened the mailing, just under 1,000 clicked through, and sales jumped. Before the mailing, sales averaged between two to four units per day. Afterward, Dessy sold 10 to 20 per day.

These are only a couple of the tests Dessy and Portent Interactive have conducted. They continue to test each mailing. You’ll likely come up with tests of your own. If you want to share your results, please email me.