Linking 101: Back to Basics

My friend and hero, Danny Sullivan, politely told me last week that there were some confusing comments in my last column, which led me to reread every column I’ve written for ClickZ. I also read through all the questions and comments I’ve received from readers in order to clarify the most common questions about linking.

How deep do search engines go into any given site when indexing content?

It depends on what brought the search engine to the site in the first place. If you submit a URL, the search engines can treat this differently than if they find your site on their own via a link on another site. A found link is deemed more worthy of crawling than a submitted link (URL). Of course, this is the old “chicken and egg” debate, as somewhere along the line someone had to submit something somewhere.

What is a deep page?

A deep page is a page that is located several directories deep on your site (e.g., http://www.yoursite.com/dir1/dir2/dir3/page.html).

Can’t I submit the deep pages from my site directly?

You can, but that doesn’t mean those deep pages will be indexed. And do not be misled by the “We successfully spidered your page” screen you get after submitting a deep page. The engine can retrieve your page without making it part of its searchable index.

If submitting deep pages doesn’t help, and I have no links to those pages from other sites, how do I ever get the search engines to index my site?

Inktomi will index any URL you want if you are willing to pay $20. If your site has 500 pages, then $10,000 gets every single one of them indexed. You don’t have to submit 500 pages. You can submit five, or one. For non-Inktomi search engines like Google, AltaVista, Excite, HotBot, WebCrawler, MSN, etc., there is no guaranteed method for ensuring that any part of your site is ever indexed. You can submit directly or be found via links to you, but in the end, there is no way to be sure you are listed.

Isn’t there some way to find out if and how many of my site’s pages have been indexed?

Yes, but not for all search engines. At AltaVista, you can enter your site’s URL (e.g., www.yourdomain.com) and find out which of your site’s pages AltaVista has indexed. Other search engines have their own terminology, and if they allow this type of search, it will be found in their Help files.

Are more links better or not?

No, more links are not better, unless all of them are high-quality links. Numbers aren’t as important as context and relevancy. It is better to have a few links from sites that are similar in content and topic to yours, a few links from the portals, and a few links from site reviewers, than to have 1,000 links on Free For All (FFA) links pages.

Is a link to my main home page better than a link to an internal page on my site?

It doesn’t matter, as ultimately the collective links to all parts of your site are more important.

I hope some of these questions and answers are helpful, and please keep your questions and comments coming. I learn something new every day and am not afraid to admit I have more to learn.

That’s why, until next time, I remain,

Eric Ward, the Link Mensch