Panama's Relevance Score Causes Pain and Gain

• Tweet
• Buffer

When phase two of Yahoo's Panama went live in early February, the new ranking algorithm almost immediately started comparing ad listings for every keyword to its neighboring listings. Only one of three things could happen for any specific keyword/creative combination within an ad group:

• The listing became more relevant that average.
  
  
• The listing became less relevant than average.
  
  
• The listing became of average relevance (although with sufficient precision, there should be no such thing as a listing with a score exactly equivalent to the average relevance).

Relevance is essentially a bell curve with an average and a standard deviation. If an ad is more relevant than average, its position would be the same as if it had raised bid price. That's because Yahoo's new algorithm multiplies predicted CTR (define) with bid price to determine a score like Google's AdRank.

Many people are confused about the way Yahoo's algorithmic change resulted in dramatic positional changes on the PPC (define) SERP (define). The easiest way to think about it is if the differences between the predicted CTR are greater than the differences between bids (on a percentage basis), you can expect Yahoo to swap positions.

Consider a case where current bids and predicted CTR (assuming top position) for the search term "taxes" are:

Under Yahoo's old auction system, the ExpressTaxRefund listing would appear at the top. Differences in bids are only about a 3 percent range. However, the differences in predicted CTR are much higher, approximately a 23 percent range. The scores used by Yahoo to determine new positions with the new algorithm are significantly affected by the CTR percentage.

In this example, it a bid of more than $1.23 is needed to take the top spot. If $1.23 happens to be above the max bid ExpressTaxRefund is willing to pay, its position will drop significantly. How far it drops depends on the overall scores of neighboring listings. The position drop for an ad listing that's significantly less relevant, as measured by predicted CTR, can be dramatic, resulting in a loss in clicks and reduced spending.

Based on the above example, when the algorithmic change kicked in, many advertisers found themselves on the relevance curve's positive side. Yet just as many listings fell to the negative side. Most often, an individual account/advertiser will have a disproportionate number of ads ranking either better or worse than average. Part of this effect is due to the strong impact of the domain name/URL in searcher preference and CTR.

If you've been blindsided by Yahoo's changes, all's not lost. Relevance and CTR aren't entirely driven by the URL. Ad creative and the right campaign structure have a lot to do with relevance. There are steps marketers should take to ensure relevance moving forward. Even if you were lucky enough to benefit from Yahoo's algorithmic change in the short term, your competition will implement these kinds of relevance-enhancing tactics. This puts any newfound campaign improvements at risk.

To enhance relevance:

• Reorganize ad groups. When Yahoo completed account migration into Panama, it automatically created ad groups when keywords shared the same ad copy. The resulting structure may not be the most relevant. Titles and descriptions often work better, and are perceived as more relevant by searchers, when they contain part or all the search keywords. Resist the temptation to use the Dynamic Keyword Insertion (DKI) tool as a way to make titles and descriptions include the search term. This results in a new form of homogeneity.
  
  
• Test new titles and descriptions. Yahoo now lets you test different ad creative. Take advantage of this feature. Your estimate of the best ad may not be the one searchers like best. Set up at least two ads and test them. It's labor intensive, so start with the highest-volume ad groups/keywords.
  
  
• Review landing pages. Though Yahoo doesn't take landing page relevance into account when calculating a score, site visitors certainly expect a relevant landing page. With a better landing page, you'll get a higher conversion rate; with a higher conversion rate, you'll be able to afford higher bids.
  
  
• Review bids. The last resort to regain position and traffic should be to raise bids. However, sometimes marketers don't take into account all the right factors in setting bids. Do Yahoo clicks from a particular keyword result in better conversions, offline sales, e-mail registrations, and phone calls? Do site visitors stay around, or leave with a greater likelihood of doing business with you in the future (branding)?

Apply all the things you've learned in one search engine to the other ones. They've become more alike than different. To keep on top of your game, embrace best practices and use each engine's technology optimally.

Join us for Search Engine Strategies April 10-13 at the Hilton New York in New York City.

Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.