Indexing and Ranking: The Chicken or the Egg?

  |  January 22, 2007   |  Comments

Should you worry about your site's crawlability before you worry about its ranking?

After my little sabbatical from corporate life, I decided to start from scratch and launch a new brand. All the business and potential business coming my way at the moment is by referral. I'm very lucky in that sense, as I hate knocking on doors. But my greatest wish is that those people who are referred don't immediately burst into incessant waffling about the technologies used to build their sites, search engine spiders, meta tags, and PageRank, ad infinitum.

I often find myself talking to people who still believe ranking at search engines is all about some technical voodoo done to Web pages. I try to explain doing some basic tech-tweaking to Web pages to get them indexed isn't difficult, but getting a decent top 10 rank is. And there's this kind of astonishment from them when they discover that instead of instantly wanting to talk about technology, I first want to discuss marketing.

There's a little discipline I have: I don't take on no-hopers. The mere thought of failure makes me queasy. So talking to anyone who seems to believe that ranking at search engines is still something you achieve via a technical process, I'm sorry, but get real!

I've had to tool up my new business. Some technology I'm having custom-built, some I've bought off the shelf. I've got keyword analysis tools, link analysis tools, a vertical directory plotter, and a link-gathering bot. I've got an XML sitemap tool. I've got bid management and analysis software. I've even got a ranking tool for those poor souls who insist on having one. In fact, I've got pretty much everything your average SEO (define)/SEM (define) firm has. (It really doesn't cost much to set up an SEO/SEM shop, does it?)

But when I look at all this technology, I can say of none of it, "Without this tool, I'd never get my clients ranked at search engines." I rely on my marketing skills for that, not technology.

Link building is a marketing exercise. It's critical for ranking at the major search engines. I often say at conferences, "Don't be a link collector. Be a business developer." I tend to consider business and marketing objectives when it comes to linking.

Critical as link building is, it's still only one important factor in the ranking mix. True, links infer authority when it comes to ranking. But does authority mean quality?

Quality is a subjective thing, and search engine end users are better placed to judge quality than a link-based relevancy algorithm. Two Web pages written on a popular symphony may be completely relevant to a query. But the page written by a first-year music student may vary immensely in information quality compared to the one written by an accomplished orchestra conductor.

This is one reason end-user- behavior data and machine-learning techniques are becoming so important. Last year, I conducted a lot of research into Microsoft's RankNet approach. Moving beyond PageRank and dynamic ranking in favor of machine learning is a big research area.

I've found nothing specific that says search engines are definitely looking at the following, but it's mentioned in the literature. There's a lot of implicit information you can get from simply monitoring a Web page's popularity. Over time, you can measure how often it's visited from toolbar data, proxy logs, and click-through data. How long do people stay on particular pages? Do they hit the "Back" button after a period, or do they leave via a link?

When it comes to ranking, we're used to hearing Google say that over 100 factors are involved (I believe it moved up to 200 at some point). And Microsoft has gone on record to say that RankNet calculates 600 features for each document.

The bigger a search engine index gets, the less it can rely on pure linkage data and text to rank documents. And the harder it becomes for us to meet the new ranking criteria. That is, unless we become smarter marketers.

A friend of mine appeared on breakfast TV in the U.K. every morning last week, promoting his new self-help product. We run a Web site together that sells his products. I've been watching the referrer data in the site logs like a hawk. The very minute he finished one of his appearances, thousands of visitors hit the site. And 90 percent of them came through search: searches for his name, his products, and generic terms.

As people poured over his personal development and self-help products, they stayed on the pages for up to 14 minutes and usually bought two products. That kind of data doesn't go unnoticed by search engines.

I've checked the personal development space both on- and offline. It's a hugely competitive area with all types of lifestyle gurus who make you feel better about yourself, help you get rich, blah, blah, blah. But if they want to get search traffic in the kind of quantities my buddy gets, they'll have to advertise and promote themselves at the same level he does.

That's why I think there's a bit of a chicken-and-egg situation that goes on sometimes with indexing and ranking. Should you worry about the crawlability of your Web site before you have an inkling of whether you stand a cat's chance in hell of ever being able to compete and rank up with the big dogs?

Join us for Search Engine Strategies in London, February 13-15, at ExCel London.

Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.

ClickZ Live Toronto Twitter Canada MD Kirstine Stewart to Keynote Toronto
ClickZ Live Toronto (May 14-16) is a new event addressing the rapidly changing landscape that digital marketers face. The agenda focuses on customer engagement and attaining maximum ROI through online marketing efforts across paid, owned & earned media. Register now and save!

ABOUT THE AUTHOR

Mike Grehan

Mike Grehan is Publisher of Search Engine Watch and ClickZ and Producer of the SES international conference series. He is the current president of global trade association SEMPO, having been elected to the board of directors in 2010.

Formerly, Mike worked as a search marketing consultant with a number of international agencies, handling such global clients as SAP and Motorola. Recognized as a leading search marketing expert, Mike came online in 1995 and is author of numerous books and white papers on the subject. He is currently in the process of writing his new book "From Search To Social: Marketing To The Connected Consumer" to be published by Wiley in 2013.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Search newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

Featured White Papers

ion Interactive 9 Strategies to Engage Your Consumers White Paper

9 Strategies to Engage Your Consumers
Online marketing apps are highly engaging - taking visitors on short, but effective, conversion-focused journeys. This white paper illustrates 9 strategies to engaging consumers through app-like experiences.

eMarketer: Redefining Mobile-Only Users: Millions Selectively Avoid the Desktop

Redefining 'Mobile-Only' Users: Millions Selectively Avoid the Desktop
A new breed of selective mobile-only consumers has emerged. What are the demos of these users and how and where can marketers reach them?

Jobs

    • Sr. Paid Search Manager
      Sr. Paid Search Manager (Bisk Education) - TampaCurrently seeking a Sr. Paid Search Manager in Tampa, FL for Bisk Education! Bisk Education is a...
    • Contact Center Professional
      Contact Center Professional (TCC: The Contact Center) - Hunt ValleyLooking to join a workforce that prides themselves on being routine and keeping...
    • Recruitment and Team Building Ambassador
      Recruitment and Team Building Ambassador (Agora Inc.) - BaltimoreAgora, www.agora-inc.com, continues to expand! In order to meet the needs of our...