The Tail That Wags the Web

Over the past two years, an innovative marketing concept has continued to meander its way through the SEM (define) community. It’s known as the long tail of search. “Wired” editor Chris Anderson deftly puts forward the argument that low-demand products can collectively make up a market share that equals or exceeds high-demand products — if the distribution channel is large enough.

The term “long tail” is a particularly descriptive approach for providing a visual representation of a Zipf or Pareto distribution graph, more commonly known as the 80/20 rule. In these graphical distributions, high-demand products and services rise upward to the left and low demand products and services gradually “tail off” to the right:

Click on thumb nail to view full-size image

A similar statistical distribution occurs when you study search referral traffic patterns. Common keywords compose the bulk of branded search referrals to form the search head, while unbranded and obscure keywords stretch out to make up the search tail. This phenomenon holds true for both paid and natural search.

In many cases, the infrequent search queries — the long tail of search represented above by the graph’s extended width — can cumulatively outnumber the more frequent search queries that compose the graph’s height. In aggregate form, the long tail keyword phrases compose the majority of search-referred traffic, as opposed to the minority of search terms that fuel the search head.

The long tail principle functions in a way contrary to how most online brands prioritize their SEM strategies. Most online merchants tend to target the search head first, optimizing their sites for the top 20 keywords and keyword phrases that drive the most traffic.

Put into the milieu of natural SEO (define), e-commerce strategists would do well to also focus on targeting the millions of unbranded niche phrases at the shallow end of the search referral stream to complement high-traffic, brand-oriented search referrals.

Generally speaking, anywhere from 80 to 95 percent of search engine referrals originate in branded search queries. These types of queries usually include the merchant’s name, derivatives of the merchant’s name, the domain name, and the URL.

For example, “northern tool,” “northern tool and equipment,” “northertool.com,” and “northern tool + [keyword phrase]” should present users with a www.northerntool.com page prominently positioned in natural search results because each and every search query is brand-specific to Northern Tool. The brand is expected to readily win these search referrals with little or no contest, so long as the site is structurally friendly to spiders.

Long tail theory suggests an expansive search referral treasure trove exists in the unbranded keyword universe. But a strong brand should do more than just yield brand-centric search traffic. It should be put into a position to win non-branded search referrals too. To do so, more than 30 pages of the site need to be optimized.

Although branded search queries drive the search head, the number of pages that receive search referral benefits is usually a diminutive figure. Studies show as few as 5 percent of a site’s pages successfully yield traffic for branded search queries.

Conversely, this leaves 95 percent of an online merchant’s site that can be leveraged to target search tail keywords and keyword phrases. The primary concern is to get those pages yielding traffic. The non-yielding hordes of pages must be optimized, because merely being crawled and indexed aren’t enough.

The dilemma that online merchants routinely face in such endeavors is the scale of optimizing non-yielding pages within their sites. Once they get past the burdensome issue of “Why should I bother?” that is.

Based on Forrester Research studies, the average retail brand spent $978,000 in 2005 on paid search marketing. With CPC (define) rates fluctuating, online marketers seek to improve diminishing returns with natural search optimization — because it won’t cost them a million bucks a year.

SEO tends to provide greater returns long after best practices have been baked into the site. And anyone who’s initiated a long tail-oriented PPC (define) campaign quickly learns his efforts won’t produce results. Long tail exact match keyword buys tend to succumb to CTR (define) malaise faster than you can say “Google AdWords.”

The key to making search’s long tail wag for your site starts with understanding which search-referred traffic triggers successful conversions, be they online transactions or sales leads. It’s critical you take a long, hard look at your Web stats before you chase that long tail.

You must understand the difference between pages that yield traffic and those that don’t. Next time, a closer look at what sort of key performance indicators (KPIs) you need to know to optimize your site for search’s long tail, as well as which SEO tactics take precedence when seeking to wag that tail.

Additional Resources

• The Long Tail blog
• Wikipedia’s ‘long tail definition
• “Chasing the Long Tail of Natural Search” (register for the whitepaper download)

Vote for your favorite product or campaign for the 2006 ClickZ Marketing Excellence Awards, October 30 through close of business (5 pm EST) on November 8. Winners will be announced on November 13.

Join us for Search Engine Strategies in Chicago, December 4-7, at the Hilton Chicago.

Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.