Why too many Web domains contain a jumble of mirrored pages, hurting SEO.
When we begin working with a new client, one of our first jobs is to conduct a monstrous evaluation of the site, yielding dozens of recommendations to address architecture and content.
Today's SEO (define) client really gets many of the things that, even two or three years ago, seemed quite vague and unintuitive. For example, fewer and fewer clients -- especially big brands -- believe they have a divine right to rankings and search traffic. They know it's a struggle to achieve and maintain search traffic, and that steady growth is the result of constant work.
In other aspects of site maintenance, however, site owners still make the same mistakes they made years ago. In some cases, they don't know they're doing it. In others, they know they're doing it but don't know it's hurting them.
As the Web became popular and a legitimate marketing expenditure, companies gobbled up domains based on their company name, products, and services -- including misspellings and nicknames. By itself, this is a sound preventive practice, because recent reports about "domainers" (companies whose sole purpose is to buy and sell domains from their portfolios) justify a creative domain philosophy.
The problem is that once those domains were secured, companies typically forgot about them due to long registration periods, cheap hosting, or personnel churn. In most cases, those domains mirrored the main site.
We always do a reverse-IP check to see what else sits on the client's IP address. Regardless of the size of the client, it's not uncommon to find 5, 10, or more mirror domains -- each reminiscent of the company's name, products, or services.
Sometimes the marketing department is shocked, having no idea the company owned those domains in the first place. Sometimes they're not surprised at all. Only a year ago, a new client told me, "Well, if one version of a site is good, I figured five must be better." He wasn't trying to trick anyone; he was simply extrapolating.
And keep in mind that reverse-IP checks catch only those sites on the same IP address. For smaller sites or companies that have experienced a great deal of growth or turnover, it's common to find multiple mirrors across the Web, hosted by different vendors in different locations.
The simple answer is to wrangle all the wild domains and drive them along Highway 301 (define) to the main site. In a few isolated, sticky cases, however, old marketing efforts have resulted in Yahoo, DMOZ, or both directory listings for several of the domains, as well as partial indexing across the board. In other words, in the absence of a strong brand association with a particular URL, which is the main site?
SEO firms keep nagging about this, but it's important. For the most part, companies still haven't figured out how to handle canonical (define) issues on their sites.
One of the two main canonical issues is "www" vs. no "www." Simply put, only one of these versions of your site should be live, and the other should redirect (via 301) to it.
In a rare but dangerous mutation of the "www" issue, one client mistakenly enabled wildcard subdomains on its site. This means any subdomain you can think of will resolve as a legitimate page. A fan linked to the site and mistakenly used ww.company.com instead of www.company.com as the URL. Due to the site's use of relative URLs throughout its navigation structure, engines crawled and indexed dozens of pages on the site with the wrong subdomain.
The other main canonical site issue is figuring out how to terminate URLs. This issue frequently manifests itself at a site's top level, with the root of the site redirecting (often via a 302 redirect) to something like /home.cfm or /index.asp, but it can also exist deeper in the site.
I often recommend that URLs terminate at the folder level, because it's a form of future-proofing. For example, a URL like www.domain.com/products/ will remain viable during a migration from .NET to PHP, but a URL like www.domain.com/products/index.aspx would have to undergo the necessary redirection process to a URL like www.domain.com/products/home.php. Certainly a 301 redirect from one URL to the other isn't bad, but it's not as good as no redirection at all.
Engines have a tough gig. Trying to crawl through a site and figure out which URLs are really just inadvertent duplicates certainly isn't easy. And by all accounts, including their own, they are getting better at it. But this is like saying that by walking upstairs, I'm getting better at being on the moon.
If it seems like SEO professionals have been harping on the same old domain management issues for too long, I agree. But until site owners listen to us and realize they need to show engines just one version of each page on the site, we'll just have to keep blaring the horns.
Join us for Search Engine Strategies on June 12-13 in Toronto.
Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.
On the heels of a fantastic event in New York City, ClickZ Live is taking the fun and learning to Toronto, June 23-25. With over 15 years' experience delivering industry-leading events, ClickZ Live offers an action-packed, educationally-focused agenda covering all aspects of digital marketing. Early Bird Rates expire May 29. Register today and save!
Erik Dafforn is the executive vice president of Intrapromote LLC, an SEO firm headquartered in Cleveland, Ohio. Erik manages SEO campaigns for clients ranging from tiny to enormous and edits Intrapromote's blog, SEO Speedwagon. Prior to joining Intrapromote in 1999, Erik worked as a freelance writer and editor. He also worked in-house as a development editor for Macmillan and IDG Books. Erik has a Bachelor's degree in English from Wabash College. Follow Erik and Intrapromote on Twitter.
Gartner Magic Quadrant for Digital Commerce
This Magic Quadrant examines leading digital commerce platforms that enable organizations to build digital commerce sites. These commerce platforms facilitate purchasing transactions over the Web, and support the creation and continuing development of an online relationship with a consumer.
Paid Search in the Mobile Era
Google reports that paid search ads are currently driving 40+ million calls per month. Cost per click is increasing, paid search budgets are growing, and mobile continues to dominate. It's time to revamp old search strategies, reimagine stale best practices, and add new layers data to your analytics.
June 10, 2015
12:00pm ET/9:00am PT