The Mixed Message of Paid Inclusion

Search engines are sending out mixed (if not downright contradictory) messages.

I remain a defender of paid inclusion. There’s an advantage to paid programs that allow site owners to expedite controlled entry into a search engine’s database. Getting the home page of a new site listed right away can benefit search engines, their users, and site owners. Similarly, the ability to feed search engines content they might typically be unable to access can be helpful to all.

Paid inclusion can be a difficult concept for search engines to “sell” to site owners, especially for crawler-based search engines. After all, aren’t they supposed to pick up pages across the Web naturally? Why should a site owner pay to make this happen? You should, say the search engines on their paid inclusion pages, because we don’t get all the pages, we don’t index them quickly, and we don’t refresh them on a regular basis.

This message might convince site owners they’d better sign up for the various paid inclusion programs, but it also works against a completely different message search engines want potential portal partners and the public to believe: They have fresh, relevant, and comprehensive listings of what’s on the Web.

Let’s examine what crawler-based search engines are saying and the mixed message paid inclusion produces. We’ll start with the pioneer of paid inclusion, Inktomi.

Inktomi

Inktomi pitches its “Search Submit” program by saying “traditional search engines” (including Inktomi):

  • Refresh their indexes less than once a month
  • Crawl only a portion of your site (usually between 30 and 50 percent)
  • May not fully leverage your metadata

On the other hand, Inktomi tells potential partners it provides “highly relevant search results,” “large, regionally specific databases chosen from a master database of two billion URLs,” and “rapid content refresh cycles.”

So which is it? An out-of-date index that gets only parts of Web sites or quality search?

FAST

At FAST we’re told we should consider the paid inclusion program resold by its partners because “typical search engines take as long as 45 days to update their listings. And, most listings include only the home page and maybe additional section pages… With FAST PartnerSite, all of your site’s key public pages are available.”

On the page targeting potential search partners, FAST says, “We are committed to providing our users with the largest and freshest catalog on the Net.”

FAST tells potential search partners it has the largest and freshest catalog, meanwhile it tells site owners most of their pages will be ignored or not frequently refreshed.

AltaVista

AltaVista explains its paid inclusion program helps end “the need to continually submit URLs to AltaVista.”

To potential users it says, “AltaVista offers the world’s fastest, most comprehensive search service.”

Why does the most comprehensive service require site owners to continually submit their URLs?

Teoma

Even new player Teoma has inconsistencies. Here’s its paid inclusion pitch: “With Site Submit, we guarantee your most important web pages will be included in the database powering Ask.com and Teoma.com. As a member of the Site Submit program, we’ll make sure that your latest, best content is always available to our users.”

Teoma is the same service that told the press, when it emerged from beta earlier this year: “Now with Teoma’s search methodology, users can feel more confident in the accuracy and relevance of their Web searches.”

Shouldn’t an accurate and relevant search service already have a site owner’s “latest, best content”?

What About Google?

Google alone among the major crawlers doesn’t operate a paid inclusion program. The company has repeatedly said it thinks such programs are unfair and would alter the quality of search results.

Given the mixed messages above, Google’s been very wise in taking this stance. The company can say it offers high quality, comprehensive, and fresh search without fear that marketing material from its paid inclusion program will be used to dispute this, as in the above examples.

Despite this, Google’s stand leads to a different conflict. One of its businesses is making company Web sites searchable. For instance, if you go to Cisco, you can search all of Cisco’s content using Google.

Google should have as complete as record as one can get of the Cisco site, as with other sites for which it provides site search services. But this complete record is not made available to those searching the Web from Google’s own site. Google prevents this, so as not to be accused of running a form of paid inclusion.

This means that Google, whose “mission is to deliver the best search experience on the Internet by making the world’s information universally accessible and useful,” finds itself in the odd situation of making some information inaccessible on its Web site.

Clean Up the Message

Those who understand how crawlers work know they do include many of the Web’s best pages. Good content will tend to be included for free.

This doesn’t deflect from paid inclusion’s advantages. If you have a product catalog locked inside of a database, paid inclusion offers the ability to get that content into crawlers, which typically forgo that type of material.

You might have pages you believe are important that the crawlers haven’t picked up, perhaps because their link analysis systems don’t deem them important. Paid inclusion allows site owners the ability to include these.

Those offering paid inclusion don’t need to sell their programs by sending conflicting messages about what they do. Explain the value without playing on fear. It’s more honest with potential customers and less troublesome to potential searchers and search partners.

Search Engine Marketers: Include your company on SEMList.com before the list launches June 24th.

Subscribe to get your daily business insights

Whitepapers

US Mobile Streaming Behavior
Whitepaper | Mobile

US Mobile Streaming Behavior

5y

US Mobile Streaming Behavior

Streaming has become a staple of US media-viewing habits. Streaming video, however, still comes with a variety of pesky frustrations that viewers are ...

View resource
Winning the Data Game: Digital Analytics Tactics for Media Groups
Whitepaper | Analyzing Customer Data

Winning the Data Game: Digital Analytics Tactics for Media Groups

5y

Winning the Data Game: Digital Analytics Tactics f...

Data is the lifeblood of so many companies today. You need more of it, all of which at higher quality, and all the meanwhile being compliant with data...

View resource
Learning to win the talent war: how digital marketing can develop its people
Whitepaper | Digital Marketing

Learning to win the talent war: how digital marketing can develop its peopl...

2y

Learning to win the talent war: how digital market...

This report documents the findings of a Fireside chat held by ClickZ in the first quarter of 2022. It provides expert insight on how companies can ret...

View resource
Engagement To Empowerment - Winning in Today's Experience Economy
Report | Digital Transformation

Engagement To Empowerment - Winning in Today's Experience Economy

1m

Engagement To Empowerment - Winning in Today's Exp...

Customers decide fast, influenced by only 2.5 touchpoints – globally! Make sure your brand shines in those critical moments. Read More...

View resource