The Science of Search Engine Rankings

Last week, I started the first in a series of training days I’m doing with “New Media Age” magazine (“NMA”) in London this year.

As with last year’s events, there was an excellent cross section of delegates. Among those represented were a huge governmental department, a medium-sized financial services company, a book token seller, a worldwide portal site, and even NMA and its conference organizing division.

I generally split my training days into modules. In the morning, I start with a seminar to get everyone, whatever level they’re coming into SEM (define) at, on the same page. Later, I split people into workgroups, and I finish with a lively Q and A session.

When it comes to answering questions, I tend to be meticulous about separating crawling and indexing issues as requiring some technical application. And ranking is usually best addressed by talking about applying savvy marketing principles (read: link-building is a marketing exercise).

Given I tend to have more delegates from marketing backgrounds than tech, the curiosity and questions are mainly around fundamentals related to ranking rather than the technical issues of crawling: “Why doesn’t my home page appear on the first results page even when I query our company name?” “How do other Web pages always appear at the top?” These are average questions from new entrants to the game.

I doubt whether I’ve ever presented at a workshop, seminar, or conference without asking a simple question at the start: Why should you be in the top 10 at search engines?

It’s not such a stupid question. The underlying principle of search engine algorithms is sorting the wheat from the chaff. If delegates looked at their competition, they could probably answer their own questions.

I’ve written many times about two main algorithms search engines use primarily for ranking. SEO (define) aficionados are fully aware of Google’s PageRank algorithm and Jon Kleinberg’s HITS algorithm. To industry newcomers, it’s usually all unknown. I like being able to point people to papers and presentations I’ve used over the years to help marketers get a better feel for what the whole ranking thing is all about.

But there are those (like myself) who are constantly searching for more advanced information and research in the search field.

Recently, I referenced “Web Dragons,” a book about information retrieval on the Web. Not a bad book at all. But following that reference, I was sent a courtesy copy of a book that, somehow, I completely missed last year.

Google’s PageRank and Beyond: The Science of Search Engine Rankings” is a fascinating, comprehensive study of search engine ranking mechanisms. And the fact that it’s endorsed by Kleinberg himself only adds more authority to it. Written by two mathematics professors, it’s not for the faint-hearted. This is very much a science book for researchers. So if the mere mention of graph theory, linear algebra, and computer code has your eyes glazing over, this probably isn’t the book for you.

But if you can wade through the math and code, it has hordes of information, theorems, and proofs about ranking, which is invaluable material to professional marketers.

I’ve come across quite a lot of the literature referenced before. But I was really interested in a section on trends and time-sensitive search. It highlights ranking blogs, which can be some of the most rapidly changing Web pages in a search engine index.

This led me to an algorithm developed by researchers at the HP Information Dynamics Lab. They rank blogs by their so-called epidemic importance. The algorithm is called iRank and is based on hyperlinks. The two main components are text analysis around links and a temporal analysis with all links being weighted by their freshness.

Although the book spent a week on my bedside table, I should stress again, it’s not an SEO/SEM book, much as the references to our industry in “Web Dragons” are mainly negative and relate to spam. However, I was very pleased to see my friend Chris Ridings described as writing the most famous and informative SEO papers on PageRank and getting a bibliographic reference to go with it.

And my pal Bob Massa gets a mention too. But it’s a black-hat type of reference that covers his case against Google, where (wrongly, I believe) he’s painted as a link-farm pioneer.

Believe me, this book really does tell you everything you ever wanted to know about search engine rankings.

Join us for Search Engine Strategies in London, February 13-15, at ExCel London.

Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.

Related reading

Brand Top Level Domains