Indexing and Ranking: The Chicken or the Egg?

  |  January 22, 2007   |  Comments

Should you worry about your site's crawlability before you worry about its ranking?

After my little sabbatical from corporate life, I decided to start from scratch and launch a new brand. All the business and potential business coming my way at the moment is by referral. I'm very lucky in that sense, as I hate knocking on doors. But my greatest wish is that those people who are referred don't immediately burst into incessant waffling about the technologies used to build their sites, search engine spiders, meta tags, and PageRank, ad infinitum.

I often find myself talking to people who still believe ranking at search engines is all about some technical voodoo done to Web pages. I try to explain doing some basic tech-tweaking to Web pages to get them indexed isn't difficult, but getting a decent top 10 rank is. And there's this kind of astonishment from them when they discover that instead of instantly wanting to talk about technology, I first want to discuss marketing.

There's a little discipline I have: I don't take on no-hopers. The mere thought of failure makes me queasy. So talking to anyone who seems to believe that ranking at search engines is still something you achieve via a technical process, I'm sorry, but get real!

I've had to tool up my new business. Some technology I'm having custom-built, some I've bought off the shelf. I've got keyword analysis tools, link analysis tools, a vertical directory plotter, and a link-gathering bot. I've got an XML sitemap tool. I've got bid management and analysis software. I've even got a ranking tool for those poor souls who insist on having one. In fact, I've got pretty much everything your average SEO (define)/SEM (define) firm has. (It really doesn't cost much to set up an SEO/SEM shop, does it?)

But when I look at all this technology, I can say of none of it, "Without this tool, I'd never get my clients ranked at search engines." I rely on my marketing skills for that, not technology.

Link building is a marketing exercise. It's critical for ranking at the major search engines. I often say at conferences, "Don't be a link collector. Be a business developer." I tend to consider business and marketing objectives when it comes to linking.

Critical as link building is, it's still only one important factor in the ranking mix. True, links infer authority when it comes to ranking. But does authority mean quality?

Quality is a subjective thing, and search engine end users are better placed to judge quality than a link-based relevancy algorithm. Two Web pages written on a popular symphony may be completely relevant to a query. But the page written by a first-year music student may vary immensely in information quality compared to the one written by an accomplished orchestra conductor.

This is one reason end-user- behavior data and machine-learning techniques are becoming so important. Last year, I conducted a lot of research into Microsoft's RankNet approach. Moving beyond PageRank and dynamic ranking in favor of machine learning is a big research area.

I've found nothing specific that says search engines are definitely looking at the following, but it's mentioned in the literature. There's a lot of implicit information you can get from simply monitoring a Web page's popularity. Over time, you can measure how often it's visited from toolbar data, proxy logs, and click-through data. How long do people stay on particular pages? Do they hit the "Back" button after a period, or do they leave via a link?

When it comes to ranking, we're used to hearing Google say that over 100 factors are involved (I believe it moved up to 200 at some point). And Microsoft has gone on record to say that RankNet calculates 600 features for each document.

The bigger a search engine index gets, the less it can rely on pure linkage data and text to rank documents. And the harder it becomes for us to meet the new ranking criteria. That is, unless we become smarter marketers.

A friend of mine appeared on breakfast TV in the U.K. every morning last week, promoting his new self-help product. We run a Web site together that sells his products. I've been watching the referrer data in the site logs like a hawk. The very minute he finished one of his appearances, thousands of visitors hit the site. And 90 percent of them came through search: searches for his name, his products, and generic terms.

As people poured over his personal development and self-help products, they stayed on the pages for up to 14 minutes and usually bought two products. That kind of data doesn't go unnoticed by search engines.

I've checked the personal development space both on- and offline. It's a hugely competitive area with all types of lifestyle gurus who make you feel better about yourself, help you get rich, blah, blah, blah. But if they want to get search traffic in the kind of quantities my buddy gets, they'll have to advertise and promote themselves at the same level he does.

That's why I think there's a bit of a chicken-and-egg situation that goes on sometimes with indexing and ranking. Should you worry about the crawlability of your Web site before you have an inkling of whether you stand a cat's chance in hell of ever being able to compete and rank up with the big dogs?

Join us for Search Engine Strategies in London, February 13-15, at ExCel London.

Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.

ClickZ Live Chicago Learn Digital Marketing Insights From Leading Brands!
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda, or register and attend one of the best ClickZ events yet!

ABOUT THE AUTHOR

Mike Grehan

Mike Grehan is currently chief marketing officer and managing director at Acronym, where he is responsible for directing thought leadership programs and cross-platform marketing initiatives, as well as developing new, innovative content marketing campaigns.

Prior to joining Acronym, Grehan was group publishing director at Incisive Media, publisher of Search Engine Watch and ClickZ, and producer of the SES international conference series. Previously, he worked as a search marketing consultant with a number of international agencies handling global clients such as SAP and Motorola. Recognized as a leading search marketing expert, Grehan came online in 1995 and is the author of numerous books and white papers on the subject and is currently in the process of writing his new book From Search to Social: Marketing to the Connected Consumer to be published by Wiley later in 2014.

In March 2010 he was elected to SEMPO's board of directors and after a year as vice president he then served two years as president and is now the current chairman.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Search newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

UPCOMING TRAINING

Featured White Papers

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.

5 Ways to Personalize Beyond the Subject Line

5 Ways to Personalize Beyond the Subject Line
82 percent of shoppers say they would buy more items from a brand if the emails they sent were more personalized. This white paper offer five tactics that will personalize your email beyond the subject line and drive real business growth.

WEBINARS

    Information currently unavailable

Resources

Jobs

    • Executive Assistant
      Executive Assistant (Agora Inc. ) - BaltimoreAgora Inc., an international publishing company, located in the Mt. Vernon district of Baltimore, MD...
    • Paid Search Specialist
      Paid Search Specialist (Boathouse, Inc.) - Waltham  Boathouse is looking for a Paid Search Specialist to work as a part of the Digital Acquisition...
    • Paid Search / Search Engine Marketing (SEM, PPC) Specialist
      Paid Search / Search Engine Marketing (SEM, PPC) Specialist (HeBS Digital) - New YorkJOB TITLE:     Paid Search / Search Engine Marketing...