Search engines are sending out mixed (if not downright contradictory) messages.
I remain a defender of paid inclusion. There's an advantage to paid programs that allow site owners to expedite controlled entry into a search engine's database. Getting the home page of a new site listed right away can benefit search engines, their users, and site owners. Similarly, the ability to feed search engines content they might typically be unable to access can be helpful to all.
Paid inclusion can be a difficult concept for search engines to "sell" to site owners, especially for crawler-based search engines. After all, aren't they supposed to pick up pages across the Web naturally? Why should a site owner pay to make this happen? You should, say the search engines on their paid inclusion pages, because we don't get all the pages, we don't index them quickly, and we don't refresh them on a regular basis.
This message might convince site owners they'd better sign up for the various paid inclusion programs, but it also works against a completely different message search engines want potential portal partners and the public to believe: They have fresh, relevant, and comprehensive listings of what's on the Web.
Let's examine what crawler-based search engines are saying and the mixed message paid inclusion produces. We'll start with the pioneer of paid inclusion, Inktomi.
Inktomi pitches its "Search Submit" program by saying "traditional search engines" (including Inktomi):
On the other hand, Inktomi tells potential partners it provides "highly relevant search results," "large, regionally specific databases chosen from a master database of two billion URLs," and "rapid content refresh cycles."
So which is it? An out-of-date index that gets only parts of Web sites or quality search?
At FAST we're told we should consider the paid inclusion program resold by its partners because "typical search engines take as long as 45 days to update their listings. And, most listings include only the home page and maybe additional section pages... With FAST PartnerSite, all of your site's key public pages are available."
On the page targeting potential search partners, FAST says, "We are committed to providing our users with the largest and freshest catalog on the Net."
FAST tells potential search partners it has the largest and freshest catalog, meanwhile it tells site owners most of their pages will be ignored or not frequently refreshed.
AltaVista explains its paid inclusion program helps end "the need to continually submit URLs to AltaVista."
To potential users it says, "AltaVista offers the world's fastest, most comprehensive search service."
Why does the most comprehensive service require site owners to continually submit their URLs?
Even new player Teoma has inconsistencies. Here's its paid inclusion pitch: "With Site Submit, we guarantee your most important web pages will be included in the database powering Ask.com and Teoma.com. As a member of the Site Submit program, we'll make sure that your latest, best content is always available to our users."
Teoma is the same service that told the press, when it emerged from beta earlier this year: "Now with Teoma's search methodology, users can feel more confident in the accuracy and relevance of their Web searches."
Shouldn't an accurate and relevant search service already have a site owner's "latest, best content"?
What About Google?
Google alone among the major crawlers doesn't operate a paid inclusion program. The company has repeatedly said it thinks such programs are unfair and would alter the quality of search results.
Given the mixed messages above, Google's been very wise in taking this stance. The company can say it offers high quality, comprehensive, and fresh search without fear that marketing material from its paid inclusion program will be used to dispute this, as in the above examples.
Despite this, Google's stand leads to a different conflict. One of its businesses is making company Web sites searchable. For instance, if you go to Cisco, you can search all of Cisco's content using Google.
Google should have as complete as record as one can get of the Cisco site, as with other sites for which it provides site search services. But this complete record is not made available to those searching the Web from Google's own site. Google prevents this, so as not to be accused of running a form of paid inclusion.
This means that Google, whose "mission is to deliver the best search experience on the Internet by making the world's information universally accessible and useful," finds itself in the odd situation of making some information inaccessible on its Web site.
Clean Up the Message
Those who understand how crawlers work know they do include many of the Web's best pages. Good content will tend to be included for free.
This doesn't deflect from paid inclusion's advantages. If you have a product catalog locked inside of a database, paid inclusion offers the ability to get that content into crawlers, which typically forgo that type of material.
You might have pages you believe are important that the crawlers haven't picked up, perhaps because their link analysis systems don't deem them important. Paid inclusion allows site owners the ability to include these.
Those offering paid inclusion don't need to sell their programs by sending conflicting messages about what they do. Explain the value without playing on fear. It's more honest with potential customers and less troublesome to potential searchers and search partners.
Search Engine Marketers: Include your company on SEMList.com before the list launches June 24th.
Join the Industry's Leading eCommerce & Direct Marketing Experts in Chicago
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 4 days and 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda and register by Friday, August 29 to take advantage of Super Saver Rates!
IBM Social Analytics: The Science Behind Social Media Marketing
80% of internet users say they prefer to connect with brands via Facebook. 65% of social media users say they use it to learn more about brands, products and services. Learn about how to find more about customers' attitudes, preferences and buying habits from what they say on social media channels.
The Multiplier Effect of Integrating Search & Social Advertising
Latest research reveals 68% higher revenue per conversion for marketers who integrate their search & social advertising. In addition to the research results, this whitepaper also outlines 5 strategies and 15 tactics you can use to better integrate your search and social campaigns.
September 17, 2014
September 23, 2014
September 30, 2014
1:00pm ET/10:00am PT