Test Versus Control, Part 2

Two weeks ago, in Part one of this series, we discussed the important role test versus control methodology plays in email marketing and looked at an example of test versus control with a monthly e-newsletter. This week, I’ll follow up with an example of test versus control with an email promotion.

Email promotions differ from e-newsletters. They are usually one-time mailings intended to sell at the moment of delivery. An email promotion shouts, “Buy This Now!” Consider it a one-time opportunity to grab the consumer’s attention and generate desired results. An email promotion with a weak subject line, weak copy, or a weak call to action is a missed sales opportunity. How can you ensure you make the most of every sales opportunity? Test versus control.

Three key variables make email promotions successful. They’re the variables we want to test:

  • Audience. Narrowly defining your target audience is the first step to success. Too often, email promotions are blasted to a huge list of email names bought on the cheap (did someone say “spam”?). Sending promotions to a house file or a purchased list of confirmed, opt-in email addresses with a stated interest in what you have to offer will yield better results and prevent your company’s reputation from being sullied.

  • Creative. Invest time and energy in creative that catches the consumer’s attention, drives her eye to the offer, and encourages her to take action. Creative applies to the sender field, subject line, design, and body copy — anything the consumer sees or reads from the moment the email reaches the inbox. Don’t overlook the design of text-based emails. Thought and talent are required to design an appealing text message using nothing more than keyboard characters.
  • Offer. An offer that resonates with consumers is the final step to success. It can be a discount, a gift with purchase, or an added value. The wording of the offer is important. If I were selling you a product regularly priced at $100, which would sound more appealing: $25 off or 25% off?

Let’s look at an example. A software industry client wants to target software developers with an offer for a hot new software development package at a discount.

Audience

The company has a house list of software developers with over 500,000 names. It built this list over the last five years by holding special events for the developer community. Each event was an opportunity to capture additional profiling data on each developer. In addition to name and email address, the house file contains valuable data such as size, type, and industry of the developer’s company; the primary development tools used; and the most frequent type of development projects. We used the profiling data to target approximately 150,000 of the developers most likely to be interested in the new software development package.

Creative

We did earlier executions for this client, testing a variety of sender fields and designs. Past tests have yielded a design template and sender field that consistently outperform test design templates and sender fields. We’re satisfied the design template and sender field provide the best possible platform for success. We did not test these elements during this campaign.

With respect to copy and subject field, we tested four separate versions based on the MICE motivators. A colleague, Steve Tingiris, introduced me to MICE. The acronym stands for the four primary motivators of human behavior: money, ideology, coercion, and ego. We wrote four versions of copy and subject lines, each appealing on one of the four primary motivators.

Offer

The retail price on the new software development package is $2,000 — not an insignificant sum. Our client approved a $500 discount.

Now, we’re ready to test. We identified eight test groups from the house file, each with 1,000 developers. We tested two variables, copy/subject line and offer, to determine which combination yields the best results.

The copy and subject fields focus on one of the four motivators. Messages for each of the motivators are as follows:

  • Money: Developers with the latest, cutting-edge development tools will make more money.

  • Ideology: The best software is built using the latest, cutting-edge development tools.
  • Coercion: Developers without the latest, cutting-edge development tools will be left behind.
  • Ego: The best developers must have the latest, cutting-edge development tools.

The offer was presented two ways: as a discount of $500 off and as a discount of 25% off.

We sent the eight different messages to the eight separate test groups. After 48 hours, we analyzed the data:

Copy/Subject Field


Offer

Sample Size

CTR (%)

Conversion Rate (%)

Money

$500 off

1,000

12.2

6.0

Money

25% off

1,000

9.3

4.1

Ideology

$500 off

1,000

7.2

1.8

Ideology

25% off

1,000

4.1

0.8

Coercion

$500 off

1,000

6.2

1.5

Coercion

25% off

1,000

3.0

0.4

Ego

$500 off

1,000

9.2

3.4

Ego

25% off

1,000

6.2

2.1

The results show the strongest combination is when the copy/subject field is focused on the money motivator and the offer is $500 off. This will be the control version of the email promotion we send to the remaining 142,000 developers from the client’s list. By using test and control, we are able to ensure the largest base of developers receive the email promotion with the greatest impact.

What if we didn’t test the promotion? Imagine a small group of very intelligent marketing professionals gathering to massage the copy, subject line, and offer on their own. Discussion would swirl about the positives and negatives of each concept. There would be politics between line employees and their seniors. Mother-in-law research would be presented as evidence. Eventually, the group would reach a final version everyone felt was the best campaign. What if the group decided on coercion and 25 percent off?

That’s why we employ test versus control methodology and analyze observed customer data.

Related reading

/IMG/550/200550/google-gmail-logo-320x198
tencent_emily-ma_featured-image
email3-1
kenneth_ning_emarsys_featured-image
<