Intersecting Search and Blogs

I don’t intimately know blogging. Over the past three months, in the wake of Google’s purchase of major blogging tool provider Pyra, I began seeking out blogs to see what the community had to say about the move.

I’ve heard Google wanted Pyra’s Blogger.com service to shore up Google’s own news search engine. By owning Blogger, Google will learn how to better index blogs and their often timely content, the theory goes.

It doesn’t hold up. Google is quite capable of indexing blog content without owning a tool provider. It doesn’t say search technology gains were a motivation for the acquisition.

Recent blogging explorations leave me with the impression there’s a reason for Google (or another company) to create a robust, blog-specific search engine. There are things it would be nice to see bloggers do to make their content more accessible to ordinary search engines.

Blogs as Online Journals

I (and others I’ve read) characterize blogs as online diaries or journals. Bloggers post observations on their blog home pages.

Observations can cover any topic: a view of a news event, personal developments (I bought a new car; here’s why it’s cool), or comments about other bloggers (Can you believe this idiot? Have you read this essay?).

It’s common to see blogs comment on other blogs. RSS distribution makes it easy for many bloggers to do this.

Over time, a blog’s home page swells with entries. These eventually are archived — each day’s, week’s, or month’s worth of entries might move to its own pages.

Not all blog are journals. Some explore a topic, then essentially construct an article on the topic. Instead of a separate page for these, they may be posted in blog format and fall under a daily entry.

Blogs: New News Media

Who cares about blogs? Bloggers certainly do. Some are borderline fanatical in their belief the medium will replace traditional journalism. My view is blogs and traditional news outlets will both thrive and survive online. Rather than view blogs as a replacement for news outlets, I see them as complementary. Reading blogs is like reading the “Letters to the Editor” page in a newspaper.

Blogs are a new “worldwide letters” system. It’s incredibly easy for anyone to get going with blogs, to share views and information. This contributes to blogs’ rising popularity.

John Hiler wrote last year about the “blogosphere,” or Web log community and content. He points out some blogs break stories faster that traditional news outlets. Hiler’s essay covers the uneasy feelings some traditional journalists and bloggers have for each other. He believes both sides benefit.

Searching the Blogosphere

It’s not difficult for bloggers to locate news content. There are excellent news search engines. How can journalists find blog content? How can the general public search this valuable information?

There are blog-style search engines. I found all could be slow or unstable. Blog content is often not search-engine friendly, making it difficult for search engines to rank blogs.

Search Engines and Blogs

A few small changes on the part of bloggers could alter this dramatically. Remember, blogs are ordinary Web pages. When a search engine indexes pages in a blog, they aren’t viewed as “blog-like” or treated differently than “ordinary” Web pages. Because of this, blog pages face the same ranking requirements as any other Web page. In “on-the-page” ranking factors, some blogs are very anti-search engine.

Lack of Titles

Take HTML title tags. If you create a Web page and want it to rank well in a search for a particular topic, it’s helpful to include terms in the title tag. Many blogs fail to do this. A blog home page is often the blogger’s name followed by “blog,” such as “Danny Sullivan’s Blog.”

Fine, if your blog is eclectic. If you maintain a stamp-collecting blog, “Danny Sullivan’s Stamp Collecting Blog” is far better if you want to help searchers find it. It’s a change that should be easy to make with most blogging tools.

Topically Focused Pages

If you want a page covering a specific topic to be easily found, focus the page on that topic. Say you make a long comment in your blog about the war in Iraq. Further content on the same page about the birth of a child, a new PDA, a comment on another blog, and so on dilute the topic you want the page to be found for.

For search engines, it’s far better if each item you blog resides on its own page. Obviously, this runs counter to how blogs are compiled. Nevertheless, it’s worth examining whether your blog tool allows you to create standalone pages with unique titles.

An example: Dane Carlson commented on why “Google Loves Weblogs” last year. Rather than post all his comments as part of his daily Web log, he used a Radio UserLand (his blogging tool) feature to create a standalone “story.” The new page deals solely with the topic, Google Loves Weblogs, and has a unique title: Dane Carlson’s Weblog :: Google Loves Weblogs.

What if he hadn’t created a separate story? The original post would share space with musings on the possible end of dentistry and upgrading to Windows XP. Dilution aside, the page for that day’s post has an unhelpful title: Dane Carlson’s Weblog :: Wednesday, February 27, 2002.

Another example comes from Microdoc News, another Radio UserLand Web log focusing mainly on Google stories.

The Microdoc site savvily arranges things so each post can click through to its own individual page, keeping the page focused on a particular topic. Unfortunately, title tags remain generic. Though Microdoc does very well with Google for many topics, how it’s displayed in listings is poor.

An article, “Alternatives to Google Referrals,” discusses building traffic via RSS feeds. It ranked well in a Google search for “alternatives to google” when I tried it, which is great. But would you click on the below title and description?

Microdoc Microtoons: 2003/03/18
/. daily link Tuesday, 18 March 2003 Alternatives to Google Referrals.
Copyright ? 2003 Verity Intellectual Properties Pty Ltd. How many /.

If Microdoc could insert a meta description tag, it might look more attractive. Google’s used this tag more since last fall. If Microdoc could use a custom HTML page title, such as “Microdoc News: Alternatives to Google,” Google would display that title. The listing would be far more attractive.

Saved by the Links

Despite on-the-page optimization flaws, link analysis helps some blogs rise in the search engines. (This is true of nonblog pages, too.)

Bloggers (anyone, really) should explore whether their publishing tool lets them make simple changes to be more search-engine friendly. The goal isn’t to change the unique way in which blogs are constructed. They’re popular because they’re easy to construct. Slight search-engine friendliness changes may well be possible.

Search Engines and Blogs: The Future

Can search engines better work with blog content? Independently ranking page portions would help. In other words, rather than looking at all words, the title, and other on-the-page factors, a system examining portions of a page, and ranks portions independently, could help.

Another improvement might be a blog searching tool. As mentioned, some new RSS search engines have emerged. They offer blog-like searching ability, joining established Web log search engines such as Daypop. These are largely labors of love, which can impact reliability. A major search engine would have resources to produce a fast, dependable service.

How about it? Will Google or another search engine acquire Daypop? Will Google create a blog or an RSS search engine?

“We definitely have talked about taking in those feeds to improve the timeliness of our search engines,” said Sergey Brin, Google cofounder and president of technology. He remains noncommittal. Google’s first priority is to stabilize Blogger’s service. Why did Google buy it?

“The definite benefits were clear. Here was very successful product but not a successful business on its own,” Brin said. “We have the ability to scale the system and monetize it with advertising [as with contextual links]. It becomes solid business. That was the clear business case.”

Beyond making Blogger a stronger service and having a new ad venue, Brin believes other synergies are in the deal. “We’ve thought of around 20 different ideas. I’m hopeful some will come to fruition, like how can we leverage the Blogger data to improve our search, how can we improve Google’s crawl of Blogger,” he said.

Blogs and RSS

The new crop of RSS search engines I covered was inspired by a post from blogger Dave Aiello, who wants a pure-RSS search engine to find more blog content. Prominent blogger Doc Searls called for a blog-only search engine that same day.

RSS isn’t a perfect solution. Not all blogs use RSS, nor is all RSS blog content. Old hands at blog indexing, such as Daypop and Blogdex, tell me they can’t depend on RSS alone. A bigger problem will be RSS popularity growing beyond blogging. When that happens, blog-specific content could get lost. Not insurmountable, but a problem to consider.

Don’t miss ClickZ’s Weblog Business Strategies in Boston, June 9-10.