The Dynamic Detection and Redirection Impact on Global SEO

Many international websites use some type of detection to redirect visitors to content that has been designed for a specific country and/or language rather than to let visitors choose on their own. While this may come from good intentions of great user experience, if not done right, it could cause some major issues for your site that will result in a huge loss of business opportunities.

First, let’s review how common detection and redirections work.

The Language Detection and Redirection

Your system decides which language version to redirect a visitor to based on the signals from the visitor. For example, when you try to access “www.somedomain.com,” if they think you prefer seeing the site in French, it redirects you to their French site, “www.somedomain.fr,” without you even asking for it. The most common signals for this action are the visitor’s IP location or the browser language preference.

The Dynamic Detection and Redirection Using IP Location

With this method, the system detects where a visitor is accessing the website using the visitor’s IP location, and redirects him/her to a Web page designed for that location. For example, if you try to access “www.google.com” in Japan, Google redirects you to its Japanese site, “www.google.co.jp.”

The Dynamic Detection and Redirection Browser Preference

This method uses a visitor’s primary language preference submitted to the server via the browser’s request for the page, and redirects a visitor to a Web page created for the primary language. For example, if you live in Switzerland, using a French language version of Firefox will have the default language set to Swiss/French, which would send the variable “http_accept_language_fr-ch” to the Web server. In a perfect scenario, the script would detect both the French language preference and the country designator for Switzerland, and then route you to the Switzerland version of the site and render the content in French.

Now, let’s look into when these detection and redirection settings may cause problems.

Search Engine Bots Are Not People

These dynamic detection and auto-redirection methods are convenient for your visitors in most cases. However, it could limit some of your international sites from being crawled and indexed by the search engines. Here’s why.

Because the search engines are not humans, the bot won’t send the signals to your server to detect the language preference when they come to your site to crawl and index the content. It means that Google and other search engines’ bots are likely to only crawl and index the default language content. EN (English) or EN-US is usually the default language content for most U.S. websites. I have found sites developed in local markets often default to the local language version, sometimes excluding the English version. More and more international websites choose to use the generic TLD, such as “.com” for their websites, and host all language and country sites in their home market. This is a reasonable solution, considering the cost and the maintenance resources for having multiple sites and hosting each site at target country. However, because the search engines don’t crawl the sites from each country, the search engines’ bots may only see the language/country site where they crawl. Again, because the search engine bots are not human, they cannot go to your language/country selection menu to choose other languages or country content to access that content.

Here is the typical disaster waiting to happen: If Google’s bot tries to crawl your site from the U.S., and you use the IP detection to serve the content, Google would only receive your U.S. content. Note that the search engines, including Google, don’t have the bots crawling from every country. When Yandex crawls your site from its U.S. server location, and you only serve U.S. or English content based on its IP location, Yandex would not see your Russian content, potentially missing out on the Russian market.

Solutions

You could just remove the dynamic detection and the redirection setting, and let people select the language/country page they would like to see. The search engine bots will be able to crawl and index all language/country pages, too.

However, if you want to keep the detection and the redirection in place, you need to set it to treat the search engine bots differently from humans. Set it so that when it determines that a requester is a search engine, exempt it from any redirection, and give it a page it wants.

Do not switch the content on the pages to give different information to the search engine. If you do, it could become a cloaking method. All you want to do is to let a search engine see the page it requested without redirecting it to specific language or country page.