Home  › Social › Social Commerce

Measuring the Online Impact of User-Generated Content

  |  July 13, 2010   |  Comments

You can measure the real success of user-generated content with A/B testing and before-and-after comparisons.

You read user-generated content, and so do your customers. They trust it more than anything a marketer can say, and it's this trust that leads them to a confident buying decision. So when you put user-generated content on your site, how do you measure the impact? How can we tell if this content is really helping the business?

Here are two ways we recommend organizations measure real success.

Do Before-and-After Comparisons

It's possible to compare key metrics - such as average order value, sales conversion, and traffic to the product page, among others - before and after user-generated content is added to the site. The key is to look at the same (or very similar) products at specific points in time over a broad period and avoid any external factors (such as promotions).

Here's an example of a before-and-after test of reviews on a classic kitchen mixer that sees predictable sales throughout the year. Key metrics would be captured for a full year before reviews were launched. Once reviews were launched, the metrics would be compared quarterly, with a full comparative analysis done at specific points in time one year after launch.

As a best practice, compare metrics using "percent of change" rather than a simple difference, so you can compare the performance of different products to one another and make additional observations. For example, using percent of change, you may see a correlation between the number of reviews and overall conversion. Every marketer should know the percent change formula, but for the sake of convenience, here it is: % of change = [(new value - old value)/old value]*100.

To check the accuracy with this method, you can compare that mixer's results with a mixer in the same category that doesn't have reviews for the same period, or you can compare it to other products in the same category. And compare it further to products in similar categories and the site overall. With a few comparison points, assuming you have a large lift for the mixer, you can confidently support the results.

There's always the question of external factors in a before-and-after analysis, but if the results are substantial, this method can work well on a product-by-product basis on established products that have sold consistently over time. It's important to take seasonality and other factors into account, too. For example, recent nationwide economic factors played a role in many sales downturns.

Run an A/B Test

While an A/B test can take time and effort to plan, it's one of the most accurate ways to measure success, when done correctly. A/B testing takes two otherwise identical groups and makes one change to one group. That group, version "A," becomes the test group, and version "B," the group without the change, becomes the control group. You then measure the performance of these two groups, and determine what impact the variable made on the results.

For example, if you wanted to test the success of an e-mail campaign, you could do so with A/B testing. You would first split your receipt list into two groups. The control group (Group A) receives a standard e-mail, and the test group (Group B) receives the same e-mail, but with the addition of review content. You then measure key performance metrics, which, in this case, might include response rate, click-through rate, and conversion. We have seen lifts in revenue per e-mail as much as 50 percent with this method. We've also seen online retailers do A/B split tests on their websites and show lifts from 10 percent to 50 percent.

Since the only difference between Group A and B is the inclusion of user-generated content, any difference in the performance metrics can be attributed to the presence of that content.

When used correctly, it is powerful, but, if done incorrectly, it can provide misleading results. To run a successful A/B test, take time to plan ahead, base your sample size and time frame to gather enough data to show clear, realistic results, and make sure to just test one variable at a time, so you can confirm that results are based on that one variable.

Other Key Points to Remember

Look for at least a three-month test window to help make sure your results are sound. While some specific campaigns may take less time to gauge results, three months is usually a good rule of thumb.

It's also important to make sure you have the right Web analytics tags and data capture methods in place. For a holistic view of how user-generated content is working for you, tag all interactions your site visitors have with this content - including online and offline encounters. And align your tagging methods with your business' overall success metrics.

Final parting thought: If a data point doesn't matter to the bottom line (or impress the CFO), think about why you're really measuring it.

ClickZ Live New York What's New for 2015?
You spoke, we listened! ClickZ Live New York (Mar 30-Apr 1) is back with a brand new streamlined agenda. Don't miss the latest digital marketing tips, tricks and tools that will make you re-think your strategy and revolutionize your marketing campaigns. Super Saver Rates are available now. Register today!

ABOUT THE AUTHOR

Sam Decker

Sam Decker is founder and CEO of Mass Relevance, the leading enterprise social curation company. He speaks and consults on digital growth strategy, based on years of experience in technology and social markets. He has written two books on word-of-mouth marketing and is an award-winning blogger (www.deckermarketing.com). As former chief marketing officer of Bazaarvoice, the market leader in hosted social commerce applications that drive sales, Sam worked to help brands present the right user-generated content at the right time in the purchase path, bringing real value to the consumer and the business. Prior to Bazaarvoice he drove Dell's customer segmentation, their customer-centricity strategy, and led Dell's consumer website, building Dell.com into the largest consumer e-commerce site at $3.5 billion in annual sales.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Social newsletters delivered right to your inbox. Subscribe today!

COMMENTS

UPCOMING EVENTS

UPCOMING TRAINING

Featured White Papers

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.

5 Ways to Personalize Beyond the Subject Line

5 Ways to Personalize Beyond the Subject Line
82 percent of shoppers say they would buy more items from a brand if the emails they sent were more personalized. This white paper offer five tactics that will personalize your email beyond the subject line and drive real business growth.

WEBINARS

    Information currently unavailable

Jobs

    • Lead Generation Specialist
      Lead Generation Specialist (The Oxford Club) - BaltimoreThe Oxford Club is seeking a talented writer/marketer to join our growing email lead-generation...
    • Health Marketing Editor
      Health Marketing Editor (Agora Inc.) - BaltimoreCome flex your intellectual muscle as part of Agora, Inc’s (http://agora-inc.com/) legal team...
    • Technical Business Analyst
      Technical Business Analyst (OmniVista Health) - BaltimoreOmniVista Health is looking to add a Technical Business Analyst to our expanding team...