What Search Marketers Can Be Thankful For

  |  November 26, 2008   |  Comments

A list of tools and events that search engine marketers should be thankful for this year.

In a few weeks, I'll publish the results of my 2008 Webmaster wish list and add a few items I'd like to see in 2009. The early results aren't great; some things I asked for just didn't materialize. But maybe it's my fault. Why should I have expected all that stuff when I never showed what I was thankful for?

Consequently, I'm anticipating my 2009 wish list with a list of things I'm thankful for, in honor of the Thanksgiving holiday this week in the U.S. The following tools, concepts, and collaborations have made my job easier this year, and I continue to use them religiously.

Common Robots Exclusion Protocols

Traditional robots.txt exclusion protocols were pretty easy to understand, but they never really kept up with the times. I'm thankful that in early 2008, Microsoft, Yahoo, and Google joined up to expand their robots exclusion protocol to honor a series of new characters and directives that the traditional robots.txt protocol doesn't address. This move represented a vast expansion of the power Webmasters have to control bot access to their content, not only with what shows up in SERPs (define) but also how it appears.

The perfect complement to this expanded robots protocol is Google's robots.txt analysis tool in Webmaster Tools. This tool is a sandbox that enables you to plug in different URL exclusion patterns, pit them against specific URLs from your site, and see exactly how Google's bot (and in theory, any major search engine crawler) will react to the code.

LinkedIn Corporate Profiles

I'm thankful that LinkedIn corporate profiles exist, of course, but more specifically, I'm excited that LinkedIn has opened them up to be crawled. A few months ago, I wrote about LinkedIn and begged that it open them up to bots, because back then you could view them only when signed in and engines would see the log-in screen when they requested the page.

Today, I'm thrilled to see that Google has indexed over 380,000 company profiles. Yahoo is taking a little longer, showing only about 18,000 profiles, and MSN has a couple hundred.

Though I called for many of these changes, don't confuse that with my taking credit for them. Assuming credit for LinkedIn cleaning up and enabling crawling of these pages is like standing outside at 5:30 a.m. and calling for the sun to come up. It was going to happen. Reputation managers, if you're not on this one yet, get there.

Keyword Tools From Engines

For a long time, I've been only semisatisfied with the various independent keyword tools on the market. Their sample sizes are too small, forcing them to perform some unholy extrapolation to predict actual demand. They're also frequently 30 to 60 days behind the current date, and they're subject to the biases (geography, etc.) inherent in their samples.

I'm thankful that engines have begun to give us a look into their coveted user behavior data. The predictive search feature of Yahoo Search Assist is a terrific look into searchers' minds. I like it because it returns results in which your initial keyword is interspersed throughout the search phrase, unlike Google Suggest, whose results all begin with your keyword, even if a more popular phrase might have your keyword at the end.

But don't count Google out, because its Insights for Search tool is another top-notch addition to official engine-based keyword intelligence. Its industry sorting, date-based search, and geographic-based sorting are terrific, but it's the "top searches" and "rising searches" that I really dig.


Compared to 10, even 5 years ago, the wealth of information and resources available to search marketers today is astounding. Unfortunately, your competitors have access to the same information you do. Of course you're a ClickZ reader, which automatically takes you one more step ahead again. And the fact that you got all the way to the end of this column is something for which I'm truly thankful, not to mention a little surprised.

Join us for Search Engine Strategies Chicago December 8-12 at the Chicago Hilton. The only major search marketing conference and expo in the Midwest will be packed with 60-plus sessions, multiple keynotes and Orion Strategy sessions, exhibitors, networking events, and more.

ClickZ Live Chicago Join the Industry's Leading eCommerce & Direct Marketing Experts in Chicago
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 4 days and 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda and register by Friday, Oct 3 to take advantage of Early Bird Rates!


Erik Dafforn

Erik Dafforn is the executive vice president of Intrapromote LLC, an SEO firm headquartered in Cleveland, Ohio. Erik manages SEO campaigns for clients ranging from tiny to enormous and edits Intrapromote's blog, SEO Speedwagon. Prior to joining Intrapromote in 1999, Erik worked as a freelance writer and editor. He also worked in-house as a development editor for Macmillan and IDG Books. Erik has a Bachelor's degree in English from Wabash College. Follow Erik and Intrapromote on Twitter.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Search newsletter delivered to you. Subscribe today!



Featured White Papers

IBM: Social Analytics - The Science Behind Social Media Marketing

IBM Social Analytics: The Science Behind Social Media Marketing
80% of internet users say they prefer to connect with brands via Facebook. 65% of social media users say they use it to learn more about brands, products and services. Learn about how to find more about customers' attitudes, preferences and buying habits from what they say on social media channels.

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising
If you're considering implementing a marketing attribution model to measure and optimize your programs, this paper is a great introduction. It also includes real-life tips from marketers who have successfully implemented attribution in their organizations.



    • Senior Paid Search & Advertising Manager
      Senior Paid Search & Advertising Manager (Smarty Had A Party) - St. LouisCompany Description: A warm, loving, [slightly wacky] startup, we view...
    • Technical Project Manager
      Technical Project Manager (Agora Inc.) - BaltimorePublishing Services, a subsidiary of Agora Publishing in Baltimore, MD is looking for a great...
    • Financial Research Associate
      Financial Research Associate (Confidential) - BaltimoreThe Financial Research Associate will be responsible for responding to research-level customer...