SEM Power Tool: The Mighty HTTP Referrer

To really tap into search engine marketing’s (SEM’s) power, search marketers have to get technical every once in a while. This is one of those times. The more you know about the HTTP referrer (also called “the referrer”), the more information you have at your fingertips.

What exactly is an HTTP referrer? When a visitor clicks from one site to the next, the browser software temporarily stores the referring site’s URL and “tells” it to the server at the destination site; this URL is the referrer.

Knowing the URL of the page visited prior to arrival on your site is powerful information in several critical ways. Site referral data may be available for analysis in one or more ways, including Web analytics software (ASP or internal log-file-based analytics), some ASP-based paid SEM campaign management technology platforms, and even some support chat software.

We’ll cover the kinds of information found and what can be extracted from your referrer data stream. The referrer can be used to determine:

• Exact keyword phrase the searcher used before clicking on your listing. If the site visited prior to yours is a search engine, the referrer usually includes the exact keyword or keyword phrase the searcher used. This allows you to very closely duplicate the SERP of that search engine at the time the visitor clicked. This ability is great for competitive analysis.
• The source portal or network site within a syndicated search network. When you’re running paid search, you can usually see which site in the network delivered the click. This can be useful if you’re considering expanding keyword purchases into MSN or Ask Jeeves. With the right technology and reports, specific portions of an Overture or Google campaign can be analyzed to make rough projections as to the success of a direct campaign with one of their channel distribution partners.
• Broad match listings refinement. If you run broad match in Google, it’s important to explore whether some popular traffic-driving phrases deserve their own AdGroups, with specific creative and landing pages. Also review the keyword phrases coming in on broad match listings in case some of them should be added to your negative match list. Negative keywords simultaneously lift campaign performance and eliminate budget waste. Google recently made it easier to apply negative match keywords to campaigns.In Overture, there are two reasons to understand the types of traffic coming in on broad match listings. First, you must find negatives. Second, you always want a standard match listing for any reasonably high traffic phrase in Overture, because standard match is always served first.
• The click’s originating site and page. If you’re running the contextual side of a search network, you can see from which site and page a click originated. This is useful for planning other media. Seek sites similar to the ones that deliver good traffic through Google’s AdSense, Kanoodle’s content channel, or Overture’s ContentMatch.
• Site and page linked to yours, including search engines and directories. It’s nice to know who links to you. There’s an organic search engine optimization (SEO) benefit in reputable inbound links, and those links send direct traffic. If the traffic flow is significant, consider seeking out similar targeted direct links from other sites.

Combine the above list to turn referrers into important data sources for both organic and paid SEM. Applying this information can result in a significant campaign efficiency boost. A client of ours used a third-party ad-tracking system that didn’t include referrer data in reports. Without it, it was hard to understand how broad match keyword listings could be adapted to perform better. A number of third-party campaign tracking platforms weren’t designed for search campaign measurement.

Referrer data is great, but not all search-generated traffic to your site will have the correct referrer, or any referrer at all, for that matter. There are several reasons for this:

• Corporate firewalls. Some corporate firewalls strip out the referrer at the raw IP traffic level.
• Personal firewalls. Norton and other personal firewall software strip out the referrer to help preserve privacy.
• Broken browser software. Some browsers don’t handle referrers correctly. This is true for some Macintosh versions of Internet Explorer.

HTTP referrer data can add depth and breadth to campaign knowledge. More important, much of the data extracted from the mighty referrer is actionable. It can result in several percentage points worth of efficiency lift for your campaign. Find a way to use referrer data to tune a campaign by working with your vendors or internal tech team. Take the data you find and develop an action plan based on what the referrer data tells you.

Keyword prices are escalating. You need every advantage to keep your campaign performing at the highest level. To learn more about HTTP referrers and other technical aspects of SEM/SEO, attend Search Engine Strategies in San Jose, August 2-5.

Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.

Nominations are open for the 2004 ClickZ Marketing Excellence Awards.