How to Measure the Effectiveness of Email Newsletter Ads

Direct marketers always want to measure the response of ads and the overall effectiveness of promotions. Marketers using direct mail have tested everything from brochure headlines and copy to whether a stamp pulls better than a postage meter.

The web is a terrific tool for direct marketers because so many aspects of online marketing can be tracked and tested. Copy testing has been done for many years to determine how well an ad pulls compared with another ad. Banner ad serving companies such as Engage have a variety of services to track how well ads perform. Only recently, however, have email newsletter publishers provided controlled tracking and testing for advertisers.

Isolate to Measure

The key to accurate testing is to isolate the factor being tested and not allow other factors to change. When testing creative elements of an ad, it’s important to keep all other variables constant. This normally means you want to show ads to the same audience at about the same time at the same position on the page. This keeps other factors from influencing the response of one ad but not the other.

One approach to copy testing has been to perform an A/B split run test in which two different ads are each shown to half of an audience. Magazines and newspapers are able to do this by creating slightly different versions of an issue, each with different ads in a particular location within the issue. Since the audience, timing, and position are the same for each ad being tested, it’s relatively easy to assess an ad’s effectiveness.

Until recently, about the only testing that could be done with ads in email newsletters was to track click-throughs for different newsletters published about the same time. This approach leads to less-certain results because different newsletters have different audiences, ad placement policies, and publish dates.

In addition, trying to discern the relationship of CTRs to demographic data can be difficult. The very general demographic data across a broad range of newsletters available from some email newsletter publishers makes this type of analysis next to impossible.

Better Testing for Email Newsletter Ads

As email newsletter advertising grows in importance, newsletter publishers are offering sophisticated tracking and testing capabilities similar to what’s been available with rented email list services.

The use of A/B split run tests in email newsletters eliminates a number of variables that can affect the accuracy of the tests.

This type of test is generally done by having the email ad server insert one version of an ad in half of the emails in a run and insert a second version in the other half. By assigning a different URL to each ad, it’s easy to determine the CTR for each version of the ad.

In addition, when the URL includes each reader’s unique profile code, each click-through logged identifies exactly which reader responded to the ad. This means that click-throughs can be tabulated against demographic data, such as in the following example:

Ad No. 1 Ad No. 2
Male 4.2% 7.4%
Female 9.7% 5.2%
Overall 8.3% 5.9%

Other Options

In addition to A/B split run tests, several other types of testing can help newsletter advertisers maximize results. Email newsletter publishers such as Sodamail can track ad click-throughs by individual subscribers that allow precise testing, such as:

  • Analyzing CTRs by reader demographic data

  • Comparing results of the various ad positions within a newsletter
  • Tracking how response rates rise and fall over time when an ad is inserted multiple times
  • Determining the optimum length for a text ad

The ability to perform these tabulations in a highly controlled environment provides marketers with several benefits. The increased accuracy of knowing how the market reacts to different creative approaches gives advertisers more confidence that their creative is working. Also, tracking click-throughs for each demographic group provides additional marketing intelligence about how different market segments respond to ads being tested.

The process of developing online ads is highly subjective and relies on the knowledge and creativity of experienced marketers. Then it’s up to market researchers to use accurate testing methods to measure market reaction to those ads — and email newsletter publishers can play an important role in that process.

Related reading