Duplicate Content: The What, Why, and How

  |  September 20, 2010   |  Comments

Some common scenarios where duplicate content arises between domains.

Duplicate content is one of the most discussed, blogged, and talked about SEO topics - well, after link building, of course. Based on Google's webmaster guidelines, "Duplicate content generally refers to substantive blocks of content within or across domains that either completely matches other content or is appreciably similar. Most of the time when we see this, it's unintentional, or at least not malicious in origin."

What exactly do "substantive" and "appreciably similar" mean? In my view, two pages can be termed "duplicate" if 30 percent or more of the page elements - title, URL, content - are similar to each other. For example, in the news/blog world, articles are often syndicated across numerous websites.

Two types of duplicate content exist: within a domain and cross-domain.

Within a Domain

This kind of duplicate content arises within the same site or domain. The most common example is a scenario like abc.com, where http://abc.com, http://www.abc.com/index.html, and http://www.abc.com all point to the same page. The solution here is simple: 301 redirect http://abc.com and http://www.abc.com/index.html to http://www.abc.com.

Duplicate content issues can also arise when the crawler can get to the same piece of content through two or more different paths. Example: shopzilla.com/digital-cameras/402/canon/259-43010/products and shopzilla.com/digital-cameras/canon+digital+cameras/402/products are very similar. In such cases, Google picks one and discards the other. If you have tons of these types of instances on your site, I suggest using a canonical tag - pick a URL that's already indexed by Google or is more relevant to users, and have the second URL point to the primary using the canonical tag. This will help in two ways:

  • Search engines will know your preferred version and will pick that.
  • Search engines will pass link juice from one version to another, thus boosting the link juice of your preferred page.

Cross-Domain

Here are a few common scenarios where duplicate content arises between domains.

Content syndications: This usually happens when one domain syndicates its content to another domain. The example below shows CNET's content syndicated on nytimes.com. According to Google, "If you syndicate your content on other sites, Google will always show the version we think is most appropriate for users in each given search, which may or may not be the version you'd prefer."

In most instances, "most appropriate for users" corresponds to greater page authority. In this case, CNET shows up in the number two spot for the query "Canon PowerShot S90," while nytimes.com is on page three of the results.

Affiliates and co-brands: Both affiliate and co-brand deals result in duplicate content issues if not done correctly. Although co-brand deals are generally a thing of the past, they do still occur. From an SEO perspective, I would stay very far away from co-brand deals, because they inevitably result in one of the sites being completely removed from the search engine results pages (SERPs).

Here are some things to keep in mind when syndicating your content or starting an affiliate program.

  • Have the site on which your content is syndicated link back to you.
  • Ask the website syndicating your content to add a "no index" tag to prevent search engines from indexing their version of the content. If you can swing this, you should probably be in business development or sales!
  • Keep the syndicated feed different from the content that's on your site. One way of doing this is to not syndicate all of the content, or have the affiliate site display results in a different order.
  • Ensure that the key SEO elements like URL structure and title and meta tags are different between your site and the affiliate.

However, if you do find that your content is being copied and this is resulting in the scrapers ranking ahead of you, you can file a DMCA with Google, Bing, and Yahoo.

Removing duplicate content from your own domain is easy, so complete that as soon as you can. Duplicate content across domains is a totally different beast, but if you play by the rules, in most cases, you should be able to tame this beast.

This column was originally published in SES Magazine in July 2010.

Tags:

ClickZ Live Toronto On the heels of a fantastic event in New York City, ClickZ Live is taking the fun and learning to Toronto, June 23-25. With over 15 years' experience delivering industry-leading events, ClickZ Live offers an action-packed, educationally-focused agenda covering all aspects of digital marketing. Register today!

ClickZ Live San Francisco Want to learn more? Join us at ClickZ Live San Francisco, Aug 10-12!
Educating marketers for over 15 years, ClickZ Live brings together industry thought leaders from the largest brands and agencies to deliver the most advanced, educational digital marketing agenda. Register today and save $500!

ABOUT THE AUTHOR

Prashant Puri

Prashant Puri is head of global SEO for Shopping.com (an eBay Inc Company). He is responsible for SEO for sites that run across five countries. He has more than eight years of online marketing experience, including stints at Yahoo and AT&T. He's built numerous sites into multimillion unique visitor sites through a combination of SEO and SEM.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Marketing newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

UPCOMING TRAINING

Featured White Papers

Gartner Magic Quadrant for Digital Commerce

Gartner Magic Quadrant for Digital Commerce
This Magic Quadrant examines leading digital commerce platforms that enable organizations to build digital commerce sites. These commerce platforms facilitate purchasing transactions over the Web, and support the creation and continuing development of an online relationship with a consumer.

Paid Search in the Mobile Era

Paid Search in the Mobile Era
Google reports that paid search ads are currently driving 40+ million calls per month. Cost per click is increasing, paid search budgets are growing, and mobile continues to dominate. It's time to revamp old search strategies, reimagine stale best practices, and add new layers data to your analytics.

Resources

Jobs

    • Copywriting & SEO Specialist
      Copywriting & SEO Specialist (HeBS Digital) - NEW YORKJOB DESCRIPTION     JOB TITLE:         ...
    • SEO Specialist
      SEO Specialist (NJM Insurance Group) - West TrentonNew Jersey Manufacturers Insurance Company is an industry leader among its peers and the largest...
    • Paid Search / Search Engine Marketing (SEM, PPC) Specialist
      Paid Search / Search Engine Marketing (SEM, PPC) Specialist (HeBS Digital) - New York  JOB TITLE:        ...