Web Analytics: Forecasting the Impact of Change

• Tweet
• Buffer

Over the past year, I've spoken and written about monetizing and prioritizing opportunities identified through analytics to ensure your efforts are focused on the right opportunities.

Part of prioritizing opportunities is forecasting how much you can increase or lower a metric. I'm often asked, "How can I determine potential lift so I can prioritize opportunities based on the greatest impact to my business?"

Before you can forecast the lift, however, you must assign a value to the desired site behavior. For example:

• The value of a lead for a lead-generation site
  
  
• The value of an additional sale on a commerce site (you may want to look at this from a profit standpoint, not just a revenue standpoint)
  
  
• The value of deflecting calls to the call center
  
  
• The value of the use of locating a store
  
  
• The value of newsletter signup or other registration
  
  
• The value of a page view, primarily for sites that primarily generate ad revenue

These are just a sampling of desired behaviors you may want to define. Step back and think about what behaviors visitors can perform on your site that positively affect your business. Yes, this is a very selfish way to look at a visit, and we always want to consider what's important based on visitor needs, but starting this way allows you to understand your business drivers' value.

Once you've identified the desired behaviors and assigned a value to each, you can forecast the changes' impact. Unfortunately, there isn't a specific equation to determine how much you can realistically lift a specific key performance indicator (KPI).

First, learn everything you can about visitor behaviors around the desired behavior you are trying to affect. We break this research up into three categories:

• Behavioral. Through Web analytics, we can understand where people are going, where they came from, where they drop out, and what might help them to convert or stop conversion.
  
  
• Attitudinal. Through surveys, attitudinal studies, or usability groups, we can begin to get more of an understanding of the behaviors we see.
  
  
• Competitive. Using competitive data from the likes of comScore, Hitwise, and Nielsen//NetRatings, we can see how our conversion or flow compares to others in the space.

When you look at this data, it's helpful to look at visitor segments. You may find what works for some visitor segments won't work for others. You don't want to change something that's already working really well for one group.

With more detail in the three categories of understanding, you should be able to better understand the problem behind the desired behaviors you are trying to improve. Start off with conservative changes.

Let's say your site focuses on generating leads and currently has a visit-to-lead conversion rate of 3 percent. You do some research (behavioral, attitudinal, and competitive) and find a few potential issues. You wouldn't want to forecast the impact for 5 percent. Instead, you select a range based on what you see in the research, then tune that over time as you run tests. You may put a range together, such as 3.25 percent to 3.75 percent in 0.1 percent increments, to understand the value. Again, you can tune this over time, but from a prioritization standpoint you want to be conservative and look at a range you think you can realistically hit through one or two tests.

A word of warning: Don't increase visit-to-lead conversion in the above example at the cost of reducing lead quality. You can surely increase conversions by giving away an iPod to every fifth person who registers. But you will most likely be driving unqualified leads, greatly reducing lead value. You must consider and measure the outcome of such things.

As you do more of these, you'll get better at determining the potential lift based on the contributing factors you see during your research. You will be able to tighten the forecast range in terms of change.

The key is to start prioritizing opportunities based on monetized values and quickly move into testing different ideas through either simple A/B tests or more advanced multivariate tests using tools like Offermatica. Forecast the potential lift you think you can realize from the different opportunities. You'll nail some and miss others, either forecasting too high or too low. That's OK. Small changes can often lead to a big result in terms of monetized value. You forecast change to help prioritize your opportunities based on the greatest impact, as well as to help get your organization to realize the potential in opportunities that are being identified and to drive them to act on those opportunities.

Shoot me an email and let me know how it goes.