Understand Search Results Pages

Most search engine results pages (SERPs) present information from a variety of sources, usually a human-based search engine, a crawler-based search engine, and a pay-for-placement (PFP) search engine. We’re even beginning to see shopping search engine results appear in “natural” search results. What does this mean for you?

Types of Search Engines

As I mentioned previously, there are different types of search engines: human-based, crawler-based, PFP, shopping, and meta.

A human-based search engine is commonly called a directory. It uses human editors to manually categorize Web sites or pages. Examples include Yahoo, Open Directory, and Business.com.

Crawler-based search engines use special software robots, called spiders or crawlers, to retrieve information from Web pages. Examples of crawler-based search engines include Google, Teoma, Inktomi, and FAST.

Crawler-based search engines perform three basic tasks:

  • Find and fetch Web pages, a process called spidering, and build lists of words and phrases found on each page.

  • Keep an index, or database, of words and phrases spiders find on each page they’re able to crawl.
  • Allow searchers to query their indexes. Crawler-based search engines try to match the words typed in a search query with the Web page most likely to contain the information searched for.

A PFP search engine guarantees top placement in its search engine and distribution network in exchange for payment. The highest bidder gets the top position. Overture, Google AdWords, eSpotting.com, Kanoodle, and FindWhat.com are all PFP search engines.

Not just any Tom, Dick, or Harry can rank for a specific term just because he pays for it. PFP search engines are still search engines. Relevancy is still important. The product or service a site offers must be applicable to the keyword purchase.

As shopping and meta search engines are unique entities, I’ll address them in future columns. But be aware they exist and can have considerable impact on your search engine marketing (SEM) campaigns.

The Structure of a SERP

As most SERPs display information from a variety of search engines, I look at how that information is displayed before I recommend an SEM campaign to my clients.

I begin with a short keyword list. I’ll type a number of keyword phrases into all of the major search portals to see which search engine dominates the results. Let’s perform a search for “organic teas” (without the quotes) on Yahoo

On the SERP, the category result at the top of the directory page states: Shopping > Herbal Teas > Organic.

I know this result comes from Yahoo’s directory. When I view the Web results and scroll down to the bottom of TOP 20 WEB RESULTS, I see the same category appear. The data tell me this might be the best category for my client’s site to be listed under. Of course, I’d perform a large number of queries to be certain this category is the most appropriate.

Any time you see the word “category” or “directory” in a SERP, you’re viewing results from a directory database.

The next listing type in the Yahoo SERP is in red capital letters: SPONSOR RESULTS. Any time you see “sponsor” or “sponsorship,” you’re looking at advertising. Yahoo’s results currently come from Overture.

Underneath the sponsored results is a category called TOP 20 WEB RESULTS, results provided from a crawler-based search engine. Yahoo’s current source is Google. Any time you see “Web pages” or “Web results” in a SERP, you’re viewing results from a crawler database.

Pay attention to the structure of a SERP to see if directory submission or advertising is necessary. Just because an ad appears at the very top of a page doesn’t mean the advertiser gets more qualified traffic than from a directory listing or natural results.

Overture listings perform slightly better than Google AdWords listings for our clients. We suspect the reason is the SERP structure. Google’s SERPs display clearly labeled ads in text boxes on the right side of the page. Overture ads on Yahoo aren’t so obvious. Ads populate prime real estate, in the center of the page two inches from the top. Less-savvy users might not realize the Yahoo sponsor results are ads.

Top Results Don’t Guarantee Top Sales

Interesting heading from a search engine marketer, isn’t it? Too many search engine marketers focus on positioning, not end results. Top positions alone don’t determine the success or failure of a Web site.

Search engine advertising can get expensive, especially when a site receives thousands of unqualified visitors. Even with “natural” search engine optimization (SEO) campaigns, we find the number one position isn’t always best. When people search for a specific product or service, they often comparison shop. The top position may get the first click but not the final sale.

This holds particularly true for shopping search engines. Many people assume an abandoned shopping cart is an indication of poor design. In reality, shopping carts provide a means for people to comparison shop.

Shoppers add items to a shopping cart to determine taxes, shipping, and handling. Then they’ll go to another site and do the same thing. Sometimes price determines the final sale. Sometimes availability does. Sometimes, the pink cashmere sweater on one site is prettier than the one on another site. The shopping cart itself might not be the reason for abandonment.

Branding experts might argue the number one position in ads, directory category, and natural search results are the best branding spots. This is where branding experts miss the boat. What impacts branding most isn’t a top search engine listing. Usability has the biggest impact on an online brand.


Before beginning any SEM campaign, review how each major search engine displays links to Web sites. Perform queries with your top keyword phrases to see how your pages might display.

If your site has had an online presence for years, use your Web analytics software (e.g., WebTrends, ClickTracks, HitBox, etc.) to determine which search engines provide your site with the most referrals. Then, perform queries at these engines.

Maybe you can decrease advertising costs and beef up your optimization campaign. Maybe an ad campaign will deliver better ROI. Reevaluate your SEM campaigns whenever a search engine partners with another. It can save your company thousands of dollars in time and expense.

Related reading

site search hp
Space Shuttle Launch