Spam Rules Require Effective Spam Police

What’s search engine spam? The two major search providers each publish standards about what they consider unacceptable. Here are Google’s and Yahoo’s.

What’s widely considered to be spam is common knowledge. There are lots of summaries of things to avoid. Authorities such as Shari Thurow (here and here), Dave Wallace, and Jill Whalen have published recently on the topic.

So why don’t people play by the rules? A few reasons:

• Each search engine’s rules differ, though they appear closer now than in the past.
• Some people have a “cheaters are winning” mentality. If they see spam slip through, they feel they should do the same (regardless of the fact a “cheat” technique may not actually be what helps a page rank well).
• Some people don’t agree with search engine rules. They may justify use of “hidden text” via cascading style sheets to make up for the fact their home pages have no text. To them, search engines are stupid for not accepting this.
• Some people simply don’t care. As far as they’re concerned, as long as a person finds something relevant to her search, who cares how it came up?

What About Standards?

A proposed solution to search engine spam is standards. The idea has floated around for ages, but gone nowhere.

Search engine marketing (SEM) pioneer Paul Bruemmer pushed for search engine optimization (SEO) certification in 1998. As I wrote then, a “rule book” doesn’t mean and end to spam.

A push in 2001 for search engine marketing standards also went nowhere.

Better Search Engine Disclosure?

Want a real solution to spam? The search engines should agree to publish lists of companies they’ve banned. That would help consumers seeking SEM firms to understand which to avoid. If they do use a banned firm, at least they were warned of the consequences of going with a rule breaker.

I’ve suggested this before. The search engines themselves have discussed the idea at various times in the past. It’s never gone forward, as the search engines seem fearful of legal action should they out-and-out label a firm a “spammer.”

So it’s with some sympathy I defend Search Engine Marketing Professional Organization (SEMPO) when it comes under fire for not working to ensure its members adhere to search engine spam guidelines.

SEMPO recently posted a FAQ explaining why it declined to do this. This makes me feel even more strongly that if the search engines aren’t brave enough to enforce their own laws, why should the onus be on a third party that doesn’t create the rules?

They Do Enforce!

Of course, search engines do police for spam. If they catch it, a page may be penalized or banned entirely. It’s not the same, nor as effective, as an offenders list for a variety of reasons:

• An offenders list ensures those who don’t rank well or aren’t listed for perfectly innocent reasons can discover the problem is not due to a spam penalty. Far too often, people assume they accidentally spammed when they haven’t. That could lead them to make unnecessary changes.
• Sometimes, spam is simply allowed to continue. Google is particularly famous for this, preferring perhaps to seek “algorithmic” solutions than to immediately react to spam reports and yank material. Google says it wants to detect overall patterns and develop more comprehensive solutions. Unfortunately, waiting periods fuel the “anything goes” fears some search marketers have when they see spam escape prosecution.
• Disclosure helps searchers, not just companies wanting to be listed. We’ve seen outcry over filtering adult content and filtering content in response to national laws. It’s been proposed search engines should disclose what they remove. But as I’ve written several times before, no search engine discloses what it’s removed for spam reasons. It’s something searchers may want to know.

Disclosure…

To expand on the last point, Google now provides two ways to learn if it’s removed material due to the Digital Millennium Copyright Act (DMCA), a U.S. law that can cause search engines to bar listings. A 2002 DMCA case involving the Church of Scientology is probably the best-known example.

Google provides DMCA takedown requests it receives to the Chilling Effects Clearinghouse. Search for Google there and you can review the material Google’s been asked to remove.

I don’t believe Yahoo provides copies of DMCA takedown requests. You can get a sense by searching for Yahoo at Chilling Effects. Ironically, this returns requests to Google that were probably bulk directed to Yahoo and other search engines.

Google will also provide on-site notification if it’s removed or suppressed material that may otherwise have shown up in results. Search for “kazaa” and at the bottom of the page, you’ll see this notice:

In response to a complaint we received under the Digital Millennium Copyright Act, we have removed 4 result(s) from this page. If you wish, you may read the DMCA complaint for these removed results.

It’s excellent disclosure (and not something I believe the other search engines do). Google not only reveals removed results, you can click to read the complaint it reacted to.

…And Lack of Disclosure

Back to spam. Were you informed material was removed from your search results? Nope. Might there be an economic incentive for search engines to ban sites? Absolutely. That may cause those companies to resort to advertising, the accusation Google came under (and strongly denied) late last year, when many sites lost rankings.

Let’s not single out Google. Long-time search engine marketer Greg Boser nearly received a standing ovation from his peers during a session at Search Engine Strategies in 2002 when he criticized search engines for needing better standards.

An issue Boser singled out was sites would get pulled, later to hear from various search engines how they could get back in via advertising or paid inclusion.

Real-Life Disclosure

Want a real-life example of the need for disclosure? NetIQ, maker of WebTrends, recently purchased rank checking tool WebPosition. Perhaps someone went to Google and searched for “webposition” to learn more about the software.

Good luck finding WebPosition’s official site. It’s not in the top results — unusual, given Google built its reputation largely on providing good navigation to official sites. WebPosition has no pages listed in Google at all. I’ve not seen Google list WebPosition’s pages for ages. It’s probably banned. But Google doesn’t confirm these things, of course. And as a searcher, it’s not disclosed to you.

Google doesn’t like the burden popular rank-checking software such as WebPosition places on its system. It warns people not to use it. Ironically, Google has no problem accepting ads for the product from WebPosition’s many resellers.

Let me stress that similar things are happening at other search engines, too. The need for better spam disclosure is universal, not Google-specific.

Protect Yourself

Better disclosure would help for so many reasons. It would help confused site owners. It would help guide those seeking the many good SEM firms that diligently try to avoid trouble and play by the rules. I’d love to see it happen.

Until then (it could be a long wait), some suggestions:

• The search engines write the rules. You operate within their borders and are subject to whatever rules they may, or may not, publish. Break them, and you can be tossed out. Learn the rules to the extent you can, assuming you want to avoid trouble.
• Outsourcing? Knowing the rules may not help if an SEM company creates euphemisms to hide what search engines don’t like. Get references. You might try contacting the search engines themselves for advice about specific companies, but don’t hold your breath waiting for a response. Do it anyway. If you’re banned, at least there’s evidence you tried to conduct due diligence with the search engine itself.
• Looking for more on spam? Search Engine Watch members can access the Search Engine Spamming page, which has a summary of things commonly considered bad, plus a compilation of articles on the topic. The Search Engine Optimization Articles and Search Engine Marketing Articles pages also compile content that touches on spam.
• A new scholarly paper out of Stanford University, “Web Spam Taxonomy,” is a nice summary, though not entirely accurate. Google stands by the fact it doesn’t issue “URL spam” penalties for URLs containing too many keywords (I wouldn’t try this, despite assurances). An example of spam in a meta keyword tag actually doesn’t look like spam at all. But only a search engine could tell you for certain. And none do.
• For fun, visit Black Hat SEO, a humorous directory of unsavory SEM pitches Aaron Wall has received. Finally, Search Engine Watch’s Outsourcing Search Engine Marketing is a compilation of advice on seeking an SEM firm.

Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.