Dissecting Google’s Supplemental Index

I’m often asked questions about Google’s supplemental results. Supplemental results are clearly marked by Google, but they so rarely rank high in the results (except for esoteric queries where the primary index may be a little underrepresented), you may never have seen one.

There’s still a lot of speculation and ambiguity surrounding the reason Google has this secondary index. And Google is responsible for most of this as it hasn’t clearly explained what it is and exactly how you end up in there. Although the engine does hint at it.

Supplemental, auxiliary, secondary, call it what you will. All you really need to know about it is this: it’s where Google puts the trash.

Of course, one person’s trash is another’s treasure. And to be fair, Google does treat it more like a recycle bin than an actual dump. If a gem gets thrown out by mistake, it can always be restored.

Some sites I’ve looked at recently have a few hundred pages in the supplemental index. Some have millions. Whatever the figure, should you be worried? Why is Google treating your pages so badly and disrespectfully?

ClickZ is an online marketing powerhouse of a site. The depth and breadth of the knowledge and news published over the years has established the site as a primary marketing resource. And Google knows that very well. But if we’re so good and our content is so great, why do we have almost 25,000 pages in Google’s trash can?

That’s a big figure. Should we be losing sleep over it? Not at all. Let’s take a peek to see what our trash looks like to Google.

There’s a way of getting Google to let you see your supplemental results. The figure isn’t always accurate because it’s a bit of a crude hack (which will probably stop working now that this column has published!). But at least you can get an idea of the type of pages being dumped.

Copy and paste this into the Google search box (there are variations on this code): site:www.clickz.com -bollocks***

To check your own pages, just swap out the ClickZ URL for your own. Now, is there anything striking about ClickZ’s dumped pages that hits you immediately?

Correct, over 90 percent are duplicate content. More than that, they’re pages with the same title tag. And almost every URL has the parameter “print&id=” included.

The duplicate content, of course, is intended for the convenience of people printing articles to read at their leisure. But to Google, duplicate content is useless. Its mission is an index full of originals. And there’s not much point in us spending time trying to get duplicates out of the supplementary index if the originals are ranking well in the primary index.

Prevention is better than cure, so a little creativity with a robots.txt file could stop Google from indexing our print-only pages. However, if the bulk of your original pages are in the supplementary index, not an innocent duplicate content issue, ask yourself a few more questions about your site, its content, and even URL structure.

You can read one of Google’s vague answers to what a supplemental result is. You won’t find anyone from Google saying the supplementary index is Google hell or anything like it. Quite the opposite, in fact. But just because it isn’t suggesting you’ve joined a penal colony doesn’t make knowing you’re never going to rank for anything worthwhile any easier.

I’ve talked about this to Google’s Matt Cutts and Adam Lasnik. Both suggest that getting links and doing some old-fashioned white-hat SEO (define) and online marketing should help get you into Google heaven.

But Cutts and Lasnik are such nice guys and so diplomatic they would never dream of telling it the way I see it. What if your site and its content are so similar to everything else out there? Or just plain old dull? Because, believe me, I’ve looked at some sites buried in the supplemental results, and they don’t even deserve to be that well indexed.

If you eliminate the main reasons cited for being in the supplementary index, such as duplicate content and long, ugly URLs, there’s only your content and linkage data to blame for being in there.

You can read as much as you can on the subject of getting out of the supplemental index all over the Web. There’s tons of advice. But read between the lines of what guys like Cutts and Lasnik say, and you may realize the problem isn’t getting out of the supplemental index. It’s not getting in.

Join us for Search Engine Strategies on June 12-13 in Toronto.

Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.