It's Not Over Till the Stragglers Convert

  |  April 17, 2012   |  Comments

Understand your visitor behavior and know what percent of conversions tend to be delayed so that you can make a plan for how you'll treat the stragglers.

A lot has been written about data collection for landing page testing. Having high enough data rates to statistically prove a winner and avoiding seasonal fluctuations in traffic are two of the many principles that are emphasized as important elements. But there's another consideration that is often overlooked and can have significant influence over how your results are interpreted: delayed conversions, or "stragglers."

For many businesses, it's not unusual for the conversion action to take place hours or days after first hitting the landing page. The important implication for testing is that you have to understand the behavior of your stragglers to ensure you are comparing apples to apples.

Power Options is a client of my company that sells subscription-based access to an advanced stock option research. The company's model is to offer an unrestricted 14-day trial without requiring any credit card information. So it engaged my company to develop a landing page that would convert more visitors into free trial subscribers.

The original free trial sign-up process involved landing on a very information-rich page, which included a lot of details about stock option trading, Power Options, and related subjects.

ash-figure1

A couple of the graphics on the landing page linked to the follow-on page, which is where visitors could register for the free trial. This page also had a lot of text as well as thumbnail screenshots of some sample software reports.

ash-figure2

Power Options had a low data rate on the landing page we were testing, so a "coarse granularity" approach and an A/B split test were used. In other words, all ideas were combined into a single best-practices redesign instead of tinkering with a specific landing page element.

In the proposed redesign, the two-page process became a single page. Among the most striking changes was a radical simplification of the page. The focus shifted to a simple description of the free trial offer, eliminating all supporting descriptions of stock option trading and the software.

ash-figure3

There was a lot of pushback on the proposed landing page because the client was convinced that this kind of radical simplification would not work with their target audience's sophistication and interest in research. This assumption formed the foundational question of the landing page test: Did the information-rich original page instill loyalty and provide value to visitors? Or did it serve to scare away prospects with its voluminous text, complicated page layout, and unclear call-to-action?

As it turns out, the answer to both questions was "yes" - to a degree. Although the redesigned (challenger) version of the page ended up having a much higher conversion rate to free trials, the baseline registration process instilled more loyalty and resulted in higher delayed conversions among the stragglers.

ash-figure4

Usually we see a clear elbow and then delayed conversions trickling in. Sometimes the slope of the delayed conversion "tail" is proportional to the height of the elbow (when displayed on a logarithmic scale like Figure 4). The percentage of delayed conversions is the same for all tested versions of the landing page. But this is not always the case.

In this test, the challenger version had an elbow at 9.5 percent and an 11 percent final conversion rate (an increase of 16 percent). The original baseline version of the page had an elbow at 5 percent and eventually peaked at 6.5 percent (an increase of 30 percent). So the original baseline had a higher percentage of delayed conversions but a lower overall conversion rate at the end of the day.

Clearly, tracking the actions of the stragglers was critical to predicting the percentage improvement accurately. If we had measured at the elbow point (one-hour delay), we would have predicted a 90 percent improvement (9.5 percent divided by 5 percent). Once we aged the data, we saw the more accurate 70 percent projection (11 percent divided by 6.5 percent). This was very close to the actual 75 percent observed revenue-per-visitor increase after the free trial was completed. Our subsequent analysis showed that the original version influenced a higher percentage of committed visitors to return later, and closed some of the performance gap against our simplified challenger over time.

So keep this lesson in mind when designing your landing page test. You've got to understand your visitor behavior and know what percent of conversions tend to be delayed so that you can make a plan for how you'll treat the stragglers. You basically have two options:

  • Ignore the stragglers. If your percentage of stragglers is very low, you could simply measure at the elbow point on the graph. In other words, you can disregard all of the conversions that happen after a certain maximum delay time (which corresponds to the elbow for your particular landing page). Since the percentage of stragglers is small, this won't change your results significantly and you'll be able to immediately compare the newest results to those from previous data collection periods.
  • "Age" your data. Age your data from the most recent data collection period to make sure that you're comparing properly against past periods. As was the case with Power Options, the conversion rate increase will usually begin to flatten out at some point. So you should wait the appropriate amount of time before looking at the conversion rate. Exactly where to make this cutoff is a judgment call, but a good rough guideline is to aim for the point at which you reach 95 percent of the final conversion rate. Depending on your business, this could range from a few days to a few weeks.

ClickZ Live San Francisco This Year's Premier Digital Marketing Event is #CZLSF
ClickZ Live San Francisco (Aug 11-14) brings together the industry's leading practitioners and marketing strategists to deliver 4 days of educational sessions and training workshops. From Data-Driven Marketing to Social, Mobile, Display, Search and Email, this year's comprehensive agenda will help you maximize your marketing efforts and ROI. Register today!

ABOUT THE AUTHOR

Tim Ash

Tim Ash is CEO of SiteTuners.com, a landing page optimization firm that offers conversion consulting, full-service guaranteed-improvement tests, and software tools to improve conversion rates. SiteTuners' AttentionWizard.com visual attention prediction tool can be used on a landing page screenshot or mock-up to quickly identify major conversion issues. He has worked with Google, Facebook, American Express, CBS, Sony Music, Universal Studios, Verizon Wireless, Texas Instruments, and Coach.

Tim is a highly-regarded presenter at SES, eMetrics, PPC Summit, Affiliate Summit, PubCon, Affiliate Conference, and LeadsCon. He is the chairperson of ConversionConference.com, the first conference focused on improving online conversions. A columnist for several publications including ClickZ, he's host of the weekly Landing Page Optimization show and podcast on WebmasterRadio.fm. His columns can be found in the Search Engine Watch archive.

He received his B.S. and M.S. during his Ph.D. studies at UC San Diego. Tim is the author of the bestselling book, "Landing Page Optimization."

Connect with Tim on Google+.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

Featured White Papers

BigDoor: The Marketers Guide to Customer Loyalty

The Marketer's Guide to Customer Loyalty
Customer loyalty is imperative to success, but fostering and maintaining loyalty takes a lot of work. This guide is here to help marketers build, execute, and maintain a successful loyalty initiative.

Marin Software: The Multiplier Effect of Integrating Search & Social Advertising

The Multiplier Effect of Integrating Search & Social Advertising
Latest research reveals 68% higher revenue per conversion for marketers who integrate their search & social advertising. In addition to the research results, this whitepaper also outlines 5 strategies and 15 tactics you can use to better integrate your search and social campaigns.

WEBINARS

Jobs

    • Interactive Product Manager
      Interactive Product Manager (Western Governors University) - Salt Lake CityWestern Governors University, one of the 20 largest universities...
    • SEO Senior Analyst
      SEO Senior Analyst (University of Phoenix (Apollo Education Group)) - San FranciscoSEO Senior Analyst   Position Summary...
    • SEM & Biddable Media Manager
      SEM & Biddable Media Manager (Kepler Group LLC) - New YorkAs an Optimization & Innovation Manager at Kepler Group, you will be on the bleeding...