Tracking Domain Consolidation with Google Webmaster Tools

With almost every SEO recommendation I provide my clients, I’m asked how long it will take before Google detects the change. The most honest answer is “it depends” which, unfortunately, is also the most unsatisfying answer.

Recently, I got involved in a project to consolidate domains that was spurred by both the Google Panda updates combined with the need to simplify maintenance and marketing. The consolidation involved 25+ sites that each ranged from hundreds to thousands of pages. As part of this effort I closely monitored Google Webmaster Tools (GWT) to determine how quickly Google reports on what it discovers.

The First Few Days

GWT showed no data in the first few days, but I watched the crawler grab pages by looking at the site’s file-based logs. So I spent this time verifying that the redirects I had set up were indeed working, i.e. I checked each old URL. I also confirmed that traffic levels to each site before the domain consolidation added up to the combined traffic to the new domain, which further confirmed that the correct redirects were in place.

End of Week 1

According to GWT all of the content (about 4,000 pages) on the new domain had been indexed, which can be viewed at Site configuration → Sitemaps → URLs submitted vs. URLs in web index. However, the site: operator in Google search showed a smaller number.

Despite being fully indexed, URLs from the old sites were still appearing in the search engine results pages (SERPs). Curiously, the URL of an image result had been updated to point to the new site. I’m sure this wouldn’t have happened so quickly if I hadn’t set up redirects for images, which is something I think many people forget to do.

GWT was now showing crawl data (see Diagnostics → Crawl Stats) and as I had hoped the pages crawled per day was in the thousands.

I also noticed that GWT was showing some inbound link data; it was counting some of the redirects from the old domains as inbound links (see Your site on the web → Links to your site → Who links the most). Majestic SEO was also reporting some inbound links in its Fresh Index while there had been no change with the Open Site Explorer data (not surprising since link data is not updated in real-time).

End of Week 2

By the end of the second week GWT was reporting impression and click data (see Your site on the web → Search queries), but it wasn’t valuable yet since there was no history to compare. More of the old sites were appearing as inbound links and a few links from other sites also appeared.

Also at this time, GWT reported the first data points for site performance (see Labs → Site performance), but again with limited data there wasn’t much to conclude. Looking at the graph GWT provided it appeared that data was captured from day one, but it just wasn’t shown until now.

Under +1 metrics → Activity I saw data start to appear. It looks like this data was also being tracked from the day of launch, but just wasn’t reported immediately.

And a quick check of Majestic SEO showed that the rate of links found started to flatten.

End of Week 3

The SERPs started to update this week with old site URLs being replaced by URLs from the new site. Most rankings remained steady, but some did drop. I noticed that a new image result appeared that had not existed in the SERPs prior to the domain consolidation.

Old sites still mostly indexed according to GWT even though the new site was appearing in the SERPs. Just goes to show you that being indexed doesn’t mean ranking.

Google, at this point, reported data on configured dynamic parameters, e.g. about 2,000 URLs monitored (see Site configuration → URL parameters). I decided to make a change to the site to eliminate some dynamic parameters by replacing the question mark in the query string with a hash mark.

Google, at this point, showed me HTML suggestions (see Diagnostics → HTML suggestions) including identifying duplicate titles and meta descriptions (something worth cleaning up in a Post-Panda world). This data may have actually appeared in the second week as I failed to check it during that time. Regardless, I’m impressed by how quickly Google is able to provide such information.

Traffic on the new domain declined a little from the previous week.

End of Week 4

Many of the old sites are now being removed from Google’s index. Some sites were hovering around 50 percent indexation while others were down to 5 percent. It’d be interesting to see if a de-indexed site could “recover” if all redirects removed, but unfortunately that isn’t possible to test in this case.

GWT is now reporting URL parameters that it has found on its own.

End of Week 5

The change I made in week three to dynamic parameters doesn’t seem to have been picked up by Google yet. The number of URLs monitored is now over 5,000 for the parameter I swapped out with a hash mark.

The Monitoring Continues

This exercise has verified what I already knew anecdotally: Google’s infrastructure changes result in faster indexing and reporting. We can’t quite see the impact from SEO changes in real-time, but knowing that we can get feedback in a week does suggest there are opportunities for interesting SEO tests.

And while the first several weeks of a site launch are critical, I have every intention of monitoring on an ongoing basis. However, with many positive signs from Google, it’s time to switch to bi-weekly or monthly monitoring so I can get back to the business of creating and promoting good content.