Email Newsletters, Tables of Contents, and Improved Engagement

Should your email newsletter include a table of contents?

I’ve developed and/or revamped literally hundreds of email newsletter titles over the course of my career. I’m a big fan of including a table of contents to help readers quickly identify and reach the content most of interest to them.

But sometimes clients are skeptical, especially if they perceive the newsletter to be short and/or easy to skim. So we do a test – if you’re a regular reader, you know that I love testing things. Because neither what I believe, nor what the client believes, matters – what matters is how readers behave and what brings us closer to meeting our goals for the email newsletter.

In this case, the goal is to drive traffic to the website. The newsletter is only sent to paying subscribers; the more readers use the resources available on the website, the more likely they are to renew. We placed the table of contents in the pre-header, at the very top of the email.

Sometimes test results are clear cut. Other times, you need to make a judgment call. In this case, a judgment call was needed. Look at the figures below and, before you read on, think about which you would declare the winner: the control or the test.

click to enlarge

In terms of unique click-through rate (CTR) and unique click-to-open rate (CTOR), there’s no statistical difference. These metrics are “unique” because they count only one click per recipient. Based on this metric the control wins, because you never make a change unless you are certain it will have a positive impact on performance.

But…

When we look at total CTR and total CTOR, we see a different story. The “total” metrics here account for all the clicks, so if a single recipient clicks on five links, that’s five clicks (this would be counted as just one click in the unique figures).

In most cases, I don’t even calculate total metrics – there are no benchmarks and it’s not a figure that matters in most cases. But this client feels that the total clicks figures are important. And I don’t disagree.

So, although the unique click figures are inconclusive, the difference in the total click metrics are statistically significant.

One more data point: look at the open rate. The control and test showed different open rates; at a 95 percent confidence level, this was a statistically significant difference. How can this be, when the from and subject lines were the same on both versions?

My theory: it has to do with the preview pane view of the email, which I’ve found can have a dramatic impact on performance. The table of contents appeared in the preview pane view of the test version of the email, but not in the control. Since most email clients block images by default, the pre-header placement of the table of contents may have encouraged more readers to enable images to view the full email.

So what did you decide? Stick with the control version or declare the test the winner and new control?

We went with the test. The difference in the total click metrics had us leaning that direction, but the increased open rate cinched it.

So, if you have an email newsletter without a table of contents, here’s your call-to-action to test adding one. And if your test results look inconclusive at first, think about the goals of your email and dig a little deeper to see if there are lesser used metrics that might help you make a decision.

Until next time,

Jeanne

Image on home page via Shutterstock.