Going Beyond FTC Paid Inclusion Disclosure Guidelines

In part two of this series on paid inclusion, I wrote that although Yahoo currently meets U.S. Federal Trade Commission (FTC) recommendations on disclosing paid inclusion, I want to see it go beyond this.

The FTC guidelines are dated. They were drafted in 2002, when paid inclusion was still fairly new and people had more faith in search engine promises that they wouldn’t boost paid inclusion rankings. The FTC’s main concern was fixing the disclosure situation with paid placement.

Today, there’s a lot less faith that paid inclusion content won’t be boosted. Because of this, disclosure beyond what the FTC requires is essential for any search engine that wants to use paid inclusion yet retain searcher trust.

Lack of Faith

Since paid inclusion was introduced, I’ve written several columns showing how paid inclusion content has sometimes managed to skyrocket to the top of search results on different search engines.

Explanations such as “blending problems” and “link flux” have been trotted out to account for these situations. I can believe and understand these. Nevertheless, they’ve been convenient problems for the paid inclusion programs to have, as they’ve helped generate revenue.

Late last year, a BusinessWeek investigation found some advertisers were convinced paid inclusion helped them rank better, further damaging any faith that rank boosting doesn’t occur.

No doubt, some advertisers may mistakenly assume they got a rank boost. For example, a person in the BusinessWeek story did paid inclusion with LookSmart and found an immediate gain with MSN.

This wasn’t because paid inclusion listings were boosted at MSN. Instead, it was because at that time, MSN placed all LookSmart listings (paid inclusion or otherwise) before Inktomi’s crawler-based backup results. In other words, it wasn’t paid inclusion URLs that were boosted. LookSmart URLs were boosted, and plenty of them didn’t come from paid inclusion.

Regardless of advertiser confusion, it’s perception that counts. If enough people believe paid inclusion provides a boost, the idea slowly gains ground. Moreover, plenty of sophisticated advertisers fully believe paid inclusion helps them rank better. I’ve talked with some at our regular search conferences.

Show, Don’t Tell

The solution is better disclosure. Let the general public discern which paid inclusion URLs show up in search results. That permits those who care to make their own assessment, rather than just having to trust a search engine’s word.

In particular, I’d like to see disclosure right within the search results. Somehow, some way, make it easy to spot paid inclusion URLs.

This needn’t be blatant. A simple icon next to a paid inclusion URL would be enough to help those who care. In my keynote earlier this year at Search Engine Strategies in New York, I offered a blue dot example. Here’s how it might look using recent Yahoo results for “kameleon remote“:

See those two blue dots after the second and fifth listings? They flag paid inclusion URLs. How do I know? Certain clues that can be found in the listing URLs (see the member version of this column for more info).

Problems With Deciphering

Certainly, anyone could just do a manual check using URL clues themselves. The Yahoo Watch search proxy makes this even easier, spotting suspected paid inclusion URL strings and putting big dollar signs in front of them.

There are problems with this, however. Daniel Brandt, who runs the Yahoo Watch search proxy, and I exchanged a number of email messages earlier this year as we tried to determine what was paid, based on URLs strings. I’ve seen similar puzzlement in search forums, such as on WebmasterWorld.com.

If it’s hard for those who live and breathe search to make these determinations, what hope is there for the average consumer? In addition, the exact strings used that identify paid inclusion URLs are subject to change. Potentially, they could be hidden entirely.

No one should have to decipher such things. A few blue dots or something similar would easily reveal the effect of paid inclusion on results. If you consistently see lots of dots, you might feel paid inclusion is weighted too heavily. If you see only some scattered ones, you might have more faith that no boosting is going on.

Yahoo Response

So how about it, Yahoo? Can we have a few blue dots?

“Obviously, we’ve had this debate internally, and there are pros and cons,” said Tim Cadogan, vice president of search for Yahoo, when I spoke with him about the issue earlier this year.

A chief reason Yahoo gives for why it doesn’t provide inline disclosure is fear of negatively influencing searchers against paid results.

“The fact that someone paid for a value-added service doesn’t have anything to do with how they are ranked. It’s not a meaningful part of the equation. Therefore, sharing that information with the user doesn’t give a lot of insight,” Cadogan said. “If we put a disclosure on each result, you can then argue about the relative ranking for eternity.”

I entirely agree. Some people will view any paid inclusion URL as tainted. If even one shows up, some will argue it may have been given an unfair boost.

Unfortunately, that’s a consequence of mixing paid inclusion URLs with free listings. There’s no way around it. But at least with disclosure, people can’t argue anything is being hidden or hard to discover.

Too Much Info? Make It an Option!

Yahoo also argues inline disclosure may provide too much information, a particular issue given the company is currently beta-testing some cleaner result presentations.

“The page is already complex, and we don’t want to overload users. It’s not clear that it would helps the vast majority of users,” Cadogan said.

I suggested making disclosure an option. For users who care, why not let them set their preferences to have an inline disclosure displayed?

To me, it’s a perfect compromise between too much information and providing information some may care about. It’s certainly better than forcing people to decipher URLs or depend on third-party sites for disclosure. Cadogan said the idea would be considered.

The disclosure could be done creatively. Yahoo’s content acquisition program (CAP) is designed to bring in content from all over the Web that might otherwise be missed. As mentioned earlier in this series, why not display pride in this by a showing a special CAP logo or a “Quality Reviewed” mark to subtly flag CAP content? That makes it a feature, though I’d still argue paid CAP content would need to be distinguished from unpaid.

Segregate Paid Inclusion?

What about completely segregating paid inclusion results from free listings? Yahoo sees disadvantages in this.

“Introducing another section would be quite confusing, I think,” Cadogan said. “In the past, I can’t think the directory and Web layers were optimal. I don’t think we’ll want to go back to something like that.”

Cadogan is referring to how Yahoo once first showed results from its human-powered directory, then results from crawling the Web. It stopped doing this in late 2002.

I agree adding a separate layer isn’t optimal. It’s certainly in Yahoo’s financial interest to have paid inclusion content mixed in with free content. However, pulling it out could also arguably hurt relevancy and, in fact, unfairly penalize some sites.

You may be wondering if the FTC has any new plans regarding paid inclusion disclosure, given all the recent news. Last year, a News.Com article found the FTC wanted the search engines to help consumers by providing as much disclosure as possible, but there were no suggested new guidelines.

I followed up with an FTC contact recently and was told the situation remains the same: There are no new actions or disclosure guidelines to speak of relating to paid inclusion.

Mixing Feeds Is The Future

Cadogan views the ability to combine data from various feeds into a single list as essential to search success. Yahoo is working to blend free feed content, paid feed content, and crawler content. Additional data may also be mixed, such as local content, product listings, news headlines, and so on.

The challenge in mixing multiple databases is it’s hard to rank everything according to the same rules. Feed content may lack the link analysis data or page formatting information search engines use to rank ordinary HTML pages. How do you come up with a system that blends two completely different data types yet somehow ranks everything properly?

For Ask Jeeves, it was a step too far. A day after Yahoo unveiled its paid inclusion program in March, Ask Jeeves made the timely announcement it was dropping its own feed-based paid inclusion. The main reason was Ask Jeeves found it too difficult to mix the different data types fairly.

It’s a challenge for Yahoo, too, but one the company wants to continue working on.

“We agree, feeds aren’t simple. That’s why we are making it our investment. We can understand why people might find it difficult. But if we don’t take on that challenge, we’re going to limit what search engines can do,” Cadogan said.

In the next part of this series, we’ll look more closely at how Yahoo’s CAP program works to take in feed content from paid and non-paid partners. We’ll also look at Google’s efforts to achieve the same goals without involving payment.

Want to discuss issues of disclosure raised in this column? You can do so in this special thread in Search Engine Watch Forums and even vote on some ideas.

In the second part of this series, I also talked about ideas of helping Webmasters get support for their free listings outside of offering paid inclusion. I’ll be revisiting this issue more in the next installment of this series. You can suggest services you’d like to see in this forum thread.

Finally, we have a third thread inviting site owners using Yahoo’s paid inclusion programs to vote on and share how it’s affected their free listings, if at all.

This column was adopted from ClickZ’s SearchEngineWatch.com. A longer, more detailed version is available to paid Search Engine Watch members.

Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.

Nominations are open for the 2004 ClickZ Marketing Excellence Awards.