It's Not Over Till the Stragglers Convert

  |  April 17, 2012   |  Comments

Understand your visitor behavior and know what percent of conversions tend to be delayed so that you can make a plan for how you'll treat the stragglers.

A lot has been written about data collection for landing page testing. Having high enough data rates to statistically prove a winner and avoiding seasonal fluctuations in traffic are two of the many principles that are emphasized as important elements. But there's another consideration that is often overlooked and can have significant influence over how your results are interpreted: delayed conversions, or "stragglers."

For many businesses, it's not unusual for the conversion action to take place hours or days after first hitting the landing page. The important implication for testing is that you have to understand the behavior of your stragglers to ensure you are comparing apples to apples.

Power Options is a client of my company that sells subscription-based access to an advanced stock option research. The company's model is to offer an unrestricted 14-day trial without requiring any credit card information. So it engaged my company to develop a landing page that would convert more visitors into free trial subscribers.

The original free trial sign-up process involved landing on a very information-rich page, which included a lot of details about stock option trading, Power Options, and related subjects.


A couple of the graphics on the landing page linked to the follow-on page, which is where visitors could register for the free trial. This page also had a lot of text as well as thumbnail screenshots of some sample software reports.


Power Options had a low data rate on the landing page we were testing, so a "coarse granularity" approach and an A/B split test were used. In other words, all ideas were combined into a single best-practices redesign instead of tinkering with a specific landing page element.

In the proposed redesign, the two-page process became a single page. Among the most striking changes was a radical simplification of the page. The focus shifted to a simple description of the free trial offer, eliminating all supporting descriptions of stock option trading and the software.


There was a lot of pushback on the proposed landing page because the client was convinced that this kind of radical simplification would not work with their target audience's sophistication and interest in research. This assumption formed the foundational question of the landing page test: Did the information-rich original page instill loyalty and provide value to visitors? Or did it serve to scare away prospects with its voluminous text, complicated page layout, and unclear call-to-action?

As it turns out, the answer to both questions was "yes" - to a degree. Although the redesigned (challenger) version of the page ended up having a much higher conversion rate to free trials, the baseline registration process instilled more loyalty and resulted in higher delayed conversions among the stragglers.


Usually we see a clear elbow and then delayed conversions trickling in. Sometimes the slope of the delayed conversion "tail" is proportional to the height of the elbow (when displayed on a logarithmic scale like Figure 4). The percentage of delayed conversions is the same for all tested versions of the landing page. But this is not always the case.

In this test, the challenger version had an elbow at 9.5 percent and an 11 percent final conversion rate (an increase of 16 percent). The original baseline version of the page had an elbow at 5 percent and eventually peaked at 6.5 percent (an increase of 30 percent). So the original baseline had a higher percentage of delayed conversions but a lower overall conversion rate at the end of the day.

Clearly, tracking the actions of the stragglers was critical to predicting the percentage improvement accurately. If we had measured at the elbow point (one-hour delay), we would have predicted a 90 percent improvement (9.5 percent divided by 5 percent). Once we aged the data, we saw the more accurate 70 percent projection (11 percent divided by 6.5 percent). This was very close to the actual 75 percent observed revenue-per-visitor increase after the free trial was completed. Our subsequent analysis showed that the original version influenced a higher percentage of committed visitors to return later, and closed some of the performance gap against our simplified challenger over time.

So keep this lesson in mind when designing your landing page test. You've got to understand your visitor behavior and know what percent of conversions tend to be delayed so that you can make a plan for how you'll treat the stragglers. You basically have two options:

  • Ignore the stragglers. If your percentage of stragglers is very low, you could simply measure at the elbow point on the graph. In other words, you can disregard all of the conversions that happen after a certain maximum delay time (which corresponds to the elbow for your particular landing page). Since the percentage of stragglers is small, this won't change your results significantly and you'll be able to immediately compare the newest results to those from previous data collection periods.
  • "Age" your data. Age your data from the most recent data collection period to make sure that you're comparing properly against past periods. As was the case with Power Options, the conversion rate increase will usually begin to flatten out at some point. So you should wait the appropriate amount of time before looking at the conversion rate. Exactly where to make this cutoff is a judgment call, but a good rough guideline is to aim for the point at which you reach 95 percent of the final conversion rate. Depending on your business, this could range from a few days to a few weeks.

ClickZ Live New York Want to learn more?
Attend ClickZ Live New York March 30 - April 1. With over 15 years' experience delivering industry-leading events, ClickZ Live brings together over 60 expert speakers to offer an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!


Tim Ash

Tim Ash is CEO of, a landing page optimization firm that offers conversion consulting, full-service guaranteed-improvement tests, and software tools to improve conversion rates. SiteTuners' visual attention prediction tool can be used on a landing page screenshot or mock-up to quickly identify major conversion issues. He has worked with Google, Facebook, American Express, CBS, Sony Music, Universal Studios, Verizon Wireless, Texas Instruments, and Coach.

Tim is a highly-regarded presenter at SES, eMetrics, PPC Summit, Affiliate Summit, PubCon, Affiliate Conference, and LeadsCon. He is the chairperson of, the first conference focused on improving online conversions. A columnist for several publications including ClickZ, he's host of the weekly Landing Page Optimization show and podcast on His columns can be found in the Search Engine Watch archive.

He received his B.S. and M.S. during his Ph.D. studies at UC San Diego. Tim is the author of the bestselling book, "Landing Page Optimization."

Connect with Tim on Google+.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!




Featured White Papers

A Buyer's Guide to Affiliate Management Software

A Buyer's Guide to Affiliate Management Software
Manage your performance marketing with the right solution. Choose a platform that will mutually empower advertisers and media partners!

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.


    • Website Optimizer - SEO, CRO, Analytics
      Website Optimizer - SEO, CRO, Analytics (Marcel Digital) - ChicagoMarcel Digital, an award winning interactive marketing agency established in 2003...
    • Director of Marketing
      Director of Marketing (Patron Technology) - New YorkDirector of Marketing We are seeking a Director of Marketing to manage and build our marketing...
    • Senior Interactive Producer
      Senior Interactive Producer (Ready Set Rocket) - New YorkWhat You'll Do As a member of our team, the Senior Producer reports directly to our...