Home  › Social › Social Commerce

Measuring the Online Impact of User-Generated Content

  |  July 13, 2010   |  Comments

You can measure the real success of user-generated content with A/B testing and before-and-after comparisons.

You read user-generated content, and so do your customers. They trust it more than anything a marketer can say, and it's this trust that leads them to a confident buying decision. So when you put user-generated content on your site, how do you measure the impact? How can we tell if this content is really helping the business?

Here are two ways we recommend organizations measure real success.

Do Before-and-After Comparisons

It's possible to compare key metrics - such as average order value, sales conversion, and traffic to the product page, among others - before and after user-generated content is added to the site. The key is to look at the same (or very similar) products at specific points in time over a broad period and avoid any external factors (such as promotions).

Here's an example of a before-and-after test of reviews on a classic kitchen mixer that sees predictable sales throughout the year. Key metrics would be captured for a full year before reviews were launched. Once reviews were launched, the metrics would be compared quarterly, with a full comparative analysis done at specific points in time one year after launch.

As a best practice, compare metrics using "percent of change" rather than a simple difference, so you can compare the performance of different products to one another and make additional observations. For example, using percent of change, you may see a correlation between the number of reviews and overall conversion. Every marketer should know the percent change formula, but for the sake of convenience, here it is: % of change = [(new value - old value)/old value]*100.

To check the accuracy with this method, you can compare that mixer's results with a mixer in the same category that doesn't have reviews for the same period, or you can compare it to other products in the same category. And compare it further to products in similar categories and the site overall. With a few comparison points, assuming you have a large lift for the mixer, you can confidently support the results.

There's always the question of external factors in a before-and-after analysis, but if the results are substantial, this method can work well on a product-by-product basis on established products that have sold consistently over time. It's important to take seasonality and other factors into account, too. For example, recent nationwide economic factors played a role in many sales downturns.

Run an A/B Test

While an A/B test can take time and effort to plan, it's one of the most accurate ways to measure success, when done correctly. A/B testing takes two otherwise identical groups and makes one change to one group. That group, version "A," becomes the test group, and version "B," the group without the change, becomes the control group. You then measure the performance of these two groups, and determine what impact the variable made on the results.

For example, if you wanted to test the success of an e-mail campaign, you could do so with A/B testing. You would first split your receipt list into two groups. The control group (Group A) receives a standard e-mail, and the test group (Group B) receives the same e-mail, but with the addition of review content. You then measure key performance metrics, which, in this case, might include response rate, click-through rate, and conversion. We have seen lifts in revenue per e-mail as much as 50 percent with this method. We've also seen online retailers do A/B split tests on their websites and show lifts from 10 percent to 50 percent.

Since the only difference between Group A and B is the inclusion of user-generated content, any difference in the performance metrics can be attributed to the presence of that content.

When used correctly, it is powerful, but, if done incorrectly, it can provide misleading results. To run a successful A/B test, take time to plan ahead, base your sample size and time frame to gather enough data to show clear, realistic results, and make sure to just test one variable at a time, so you can confirm that results are based on that one variable.

Other Key Points to Remember

Look for at least a three-month test window to help make sure your results are sound. While some specific campaigns may take less time to gauge results, three months is usually a good rule of thumb.

It's also important to make sure you have the right Web analytics tags and data capture methods in place. For a holistic view of how user-generated content is working for you, tag all interactions your site visitors have with this content - including online and offline encounters. And align your tagging methods with your business' overall success metrics.

Final parting thought: If a data point doesn't matter to the bottom line (or impress the CFO), think about why you're really measuring it.

ClickZ Live Toronto On the heels of a fantastic event in New York City, ClickZ Live is taking the fun and learning to Toronto, June 23-25. With over 15 years' experience delivering industry-leading events, ClickZ Live offers an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!

ClickZ Live San Francisco Want to learn more? Join us at ClickZ Live San Francisco, Aug 10-12!
Educating marketers for over 15 years, ClickZ Live brings together industry thought leaders from the largest brands and agencies to deliver the most advanced, educational digital marketing agenda. Register today and save $500!


Sam Decker

Sam Decker is founder and CEO of Mass Relevance, the leading enterprise social curation company. He speaks and consults on digital growth strategy, based on years of experience in technology and social markets. He has written two books on word-of-mouth marketing and is an award-winning blogger (www.deckermarketing.com). As former chief marketing officer of Bazaarvoice, the market leader in hosted social commerce applications that drive sales, Sam worked to help brands present the right user-generated content at the right time in the purchase path, bringing real value to the consumer and the business. Prior to Bazaarvoice he drove Dell's customer segmentation, their customer-centricity strategy, and led Dell's consumer website, building Dell.com into the largest consumer e-commerce site at $3.5 billion in annual sales.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Social newsletters delivered right to your inbox. Subscribe today!



Featured White Papers

Gartner Magic Quadrant for Digital Commerce

Gartner Magic Quadrant for Digital Commerce
This Magic Quadrant examines leading digital commerce platforms that enable organizations to build digital commerce sites. These commerce platforms facilitate purchasing transactions over the Web, and support the creation and continuing development of an online relationship with a consumer.

Paid Search in the Mobile Era

Paid Search in the Mobile Era
Google reports that paid search ads are currently driving 40+ million calls per month. Cost per click is increasing, paid search budgets are growing, and mobile continues to dominate. It's time to revamp old search strategies, reimagine stale best practices, and add new layers data to your analytics.




    • GREAT Campaign Project Coordinator
      GREAT Campaign Project Coordinator (British Consulate-General, New York) - New YorkThe GREAT Britain Campaign is seeking an energetic and creative...
    • Paid Search Senior Account Manager
      Paid Search Senior Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring a strategic Paid Search Senior Account Manager...
    • Paid Search Account Manager
      Paid Search Account Manager (Hanapin Marketing) - BloomingtonHanapin Marketing is hiring an experienced Paid Search Account Manager to...