What Search Marketers Can Be Thankful For

  |  November 26, 2008   |  Comments

A list of tools and events that search engine marketers should be thankful for this year.

In a few weeks, I'll publish the results of my 2008 Webmaster wish list and add a few items I'd like to see in 2009. The early results aren't great; some things I asked for just didn't materialize. But maybe it's my fault. Why should I have expected all that stuff when I never showed what I was thankful for?

Consequently, I'm anticipating my 2009 wish list with a list of things I'm thankful for, in honor of the Thanksgiving holiday this week in the U.S. The following tools, concepts, and collaborations have made my job easier this year, and I continue to use them religiously.

Common Robots Exclusion Protocols

Traditional robots.txt exclusion protocols were pretty easy to understand, but they never really kept up with the times. I'm thankful that in early 2008, Microsoft, Yahoo, and Google joined up to expand their robots exclusion protocol to honor a series of new characters and directives that the traditional robots.txt protocol doesn't address. This move represented a vast expansion of the power Webmasters have to control bot access to their content, not only with what shows up in SERPs (define) but also how it appears.

The perfect complement to this expanded robots protocol is Google's robots.txt analysis tool in Webmaster Tools. This tool is a sandbox that enables you to plug in different URL exclusion patterns, pit them against specific URLs from your site, and see exactly how Google's bot (and in theory, any major search engine crawler) will react to the code.

LinkedIn Corporate Profiles

I'm thankful that LinkedIn corporate profiles exist, of course, but more specifically, I'm excited that LinkedIn has opened them up to be crawled. A few months ago, I wrote about LinkedIn and begged that it open them up to bots, because back then you could view them only when signed in and engines would see the log-in screen when they requested the page.

Today, I'm thrilled to see that Google has indexed over 380,000 company profiles. Yahoo is taking a little longer, showing only about 18,000 profiles, and MSN has a couple hundred.

Though I called for many of these changes, don't confuse that with my taking credit for them. Assuming credit for LinkedIn cleaning up and enabling crawling of these pages is like standing outside at 5:30 a.m. and calling for the sun to come up. It was going to happen. Reputation managers, if you're not on this one yet, get there.

Keyword Tools From Engines

For a long time, I've been only semisatisfied with the various independent keyword tools on the market. Their sample sizes are too small, forcing them to perform some unholy extrapolation to predict actual demand. They're also frequently 30 to 60 days behind the current date, and they're subject to the biases (geography, etc.) inherent in their samples.

I'm thankful that engines have begun to give us a look into their coveted user behavior data. The predictive search feature of Yahoo Search Assist is a terrific look into searchers' minds. I like it because it returns results in which your initial keyword is interspersed throughout the search phrase, unlike Google Suggest, whose results all begin with your keyword, even if a more popular phrase might have your keyword at the end.

But don't count Google out, because its Insights for Search tool is another top-notch addition to official engine-based keyword intelligence. Its industry sorting, date-based search, and geographic-based sorting are terrific, but it's the "top searches" and "rising searches" that I really dig.


Compared to 10, even 5 years ago, the wealth of information and resources available to search marketers today is astounding. Unfortunately, your competitors have access to the same information you do. Of course you're a ClickZ reader, which automatically takes you one more step ahead again. And the fact that you got all the way to the end of this column is something for which I'm truly thankful, not to mention a little surprised.

Join us for Search Engine Strategies Chicago December 8-12 at the Chicago Hilton. The only major search marketing conference and expo in the Midwest will be packed with 60-plus sessions, multiple keynotes and Orion Strategy sessions, exhibitors, networking events, and more.

ClickZ Live Toronto Twitter Canada MD Kirstine Stewart to Keynote Toronto
ClickZ Live Toronto (May 14-16) is a new event addressing the rapidly changing landscape that digital marketers face. The agenda focuses on customer engagement and attaining maximum ROI through online marketing efforts across paid, owned & earned media. Register now and save!


Erik Dafforn

Erik Dafforn is the executive vice president of Intrapromote LLC, an SEO firm headquartered in Cleveland, Ohio. Erik manages SEO campaigns for clients ranging from tiny to enormous and edits Intrapromote's blog, SEO Speedwagon. Prior to joining Intrapromote in 1999, Erik worked as a freelance writer and editor. He also worked in-house as a development editor for Macmillan and IDG Books. Erik has a Bachelor's degree in English from Wabash College. Follow Erik and Intrapromote on Twitter.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Search newsletter delivered to you. Subscribe today!



Featured White Papers

ion Interactive Marketing Apps for Landing Pages White Paper

Marketing Apps for Landing Pages White Paper
Marketing apps can elevate a formulaic landing page into a highly interactive user experience. Learn how to turn your static content into exciting marketing apps.

eMarketer: Redefining Mobile-Only Users: Millions Selectively Avoid the Desktop

Redefining 'Mobile-Only' Users: Millions Selectively Avoid the Desktop
A new breed of selective mobile-only consumers has emerged. What are the demos of these users and how and where can marketers reach them?


    • Contact Center Professional
      Contact Center Professional (TCC: The Contact Center) - Hunt ValleyLooking to join a workforce that prides themselves on being routine and keeping...
    • Recruitment and Team Building Ambassador
      Recruitment and Team Building Ambassador (Agora Inc.) - BaltimoreAgora, www.agora-inc.com, continues to expand! In order to meet the needs of our...
    • Design and Publishing Specialist
      Design and Publishing Specialist (Bonner and Partners) - BaltimoreIf you’re a hungry self-starter, creative, organized and have an extreme...