Microsoft Unveils Its New Search Engine -- at Last!

After months of speculation and two “preview” releases, Microsoft unwraps its new MSN search engine, the first major competitor to join Web search major leagues in nearly a year.

The new engine, available at beta.search.msn.com is an algorithmic search engine Microsoft engineers built from scratch. “This is our new engine that we’ve built from the ground up,” said Justin Osmer, product manager for MSN Search. Released in beta form, it’s expected to replace Yahoo search results still used at MSN sometime later this year or early in 2005.

Supported by an index of 5 billion pages, the new engine is comparable to Google and Yahoo in scope and, for most of my initial tests, relevancy. Microsoft has acknowledged being “as good as” its competitors is the entry price for the Web search game.

The interface is clean and sparse, as are SERPs (define). MSN Search now offers cached links to copies of pages its crawler fetched. Notably, for some pages, a date is also displayed next to the “Cached page” link. Microsoft says this date is an estimate of when the page last changed. In many cases, it’s the date the page was crawled. You can see the crawl date for all pages by clicking through to the cached version of the page.

Osmer says a comprehensive crawling effort is central to MSN Search’s success. MSN is crawling some pages daily, some weekly, and some monthly, “which we believe is more frequently than some of our competitors,” says Osmer.

New and Different Features

To set itself apart from the pack, the new MSN Search includes features that differ from those found on Google, Yahoo, and Ask Jeeves. The most prominent is a “Near Me” button next to the “Search” button. Clicking the Near Me button effectively runs a local search for the query. In this release, the Near Me feature works only for U.S. searchers.

By default, your browser’s IP address is used to determine your location. You can override this by explicitly entering a location using the Settings command. The Near Me function works quite well, primarily because Microsoft has tagged every Web page containing geographic information in its index. The company uses what it calls an “overlapping tiles model,” starting with Zip Code, then neighborhood, region, city, state, and country information, if available.

In addition, MSN Search incorporates additional non-Web sources of information in results on a query-specific basis. For example, queries for factual information get “direct answers” from Microsoft’s Encarta encyclopedia. For music-related queries, inline results from MSN Music appear, with a link to further information, downloads, and so on.

Microsoft also offers an interesting hybrid approach to customization and advanced search. Clicking the “Search Builder” link opens a window beneath the search form that provides explicit controls over many query variables. Most of these controls are similar to those hidden away on the advanced search page of other search engines. Making them available from any search form is a nice touch that should encourage more use of these refinement tools.

“The point is trying to allow the average person to be able to build relatively complex queries relatively easily,” said Osmer.

The first tool allows you to easily modify a query by adding terms or an exact phrase, or excluding terms from the search. You can also limit a search by domain; country or region; language; or to pages linked to a particular URL. Search Builder automatically formats your query with the appropriate syntax, even nesting queries in parentheses when appropriate.

The final refinement tool is MSN Search’s coolest feature, allowing you to control result ranking using sliders. Three sliders are available. One lets you select the degree of match between your search terms and SERPs, from an exact match to an approximate one. The second lets you specify page popularity, from very to less popular. The final slider controls freshness of results, from recently updated to static pages.

Using all three sliders in combination produces a remarkably wide range of results for the same query. For some types of searches, it can be extremely useful. In other cases, results become, well, just bizarre. I love the idea of giving searchers more control over results and I like the sliders, but this particular feature needs refinement before it will catch on.

Additionally, image search is new but not unique. MSN Search partnered with Picsearch to provide access to over 400 million images. Osmer says MSN Search starts with the Picsearch database and adds its own tweaks. At this point, results from MSN Search and Picsearch are nearly identical for the test queries I ran.

The quality of image search results is reasonably high, especially when compared to an image search in Google, which seems to have faltered lately. Among the majors, Yahoo’s recently enhanced image search database seems to be the clear leader in this area, at least for now.

Customization Options

Through the “Settings” link, you can set several preferences for search results. In addition to English, the MSN Search interface is available in 10 other languages, mostly European. You can specify which languages are to be included in results.

Other settings allow you to change the number of search results from the default of 10 to 15, 30, 50, or 100 per page.

Safe search is new. Moderate filtering is the default setting, screening out only sexually explicit images. Strict filtering will eliminate both explicit images and text. You can turn the safe search filter off, as well.

Users can influence the number of results displayed from a particular site. Results from the same site are grouped, with a maximum of two displayed by default. You can change this to display one, two, or three results.

Desktop Search: Missing, But Coming Soon

Contrary to what’s been widely reported in the media, Microsoft will release a desktop search application before year’s end. It won’t be the application that’s part of the Longhorn Windows upgrade, but a separate application that will integrate with MSN Search. I’ve seen a demo of the desktop search application and am impressed with its capabilities, but a nondisclosure agreement prohibits me from writing more until it’s actually released.

Microsoft has a number of other planned enhancements that will be released before the end of the year. These cover a wide range of both Web and desktop content, including a blog search feature, something no major search engine has yet done, despite Google’s purchase of Blogger and Yahoo’s push toward indexing RSS (define) feeds and making them easily available via My Yahoo.

These enhancements will almost certainly strengthen the appeal of the MSN Search application.

A Google Killer? Not.

Over the past few days, as rumors of MSN’s search launch were widely reported, speculation focused on whether Microsoft’s entry into the fray signals the end of Google’s Web search domination. Not likely, and not because MSN’s search engine doesn’t have the technical chops to compete with Google. It does, and Microsoft seems truly to have “got religion” when it comes to Web search.

But Google isn’t going to stand still, as we saw with last week’s stealth increase in Google’s index to a reported 8 billion-plus pages. This means Google is likely working with a full index of over 10 billion items, roughly twice the size of Microsoft’s Web index.

Google seems to have a lock on searchers’ mental shelf space, at least for now. Rightly or wrongly, many people automatically turn to Google for search. Microsoft will have to do much more than simply launch this initial, highly laudable foray to change the search behavior of the masses.

I think it will, by both improving this initial beta version and gradually folding some of the seriously cool research projects cooking in Microsoft Research labs into the search engine over the next couple years. Let’s not forget Yahoo and Ask Jeeves, also diligently improving their own search engines.

The bottom line for searchers is we’ve now got four world-class “voices,” as Danny Sullivan likes to call them, working to outdo not only their competitors but also themselves. And we’ve got dozens of smaller players developing innovative new tools and approaches that will likely be absorbed, or copied, by the big guys.

Far from being a Google killer, MSN Search is instead a welcome new alternative for searchers and a catalyst for sparking further improvements and innovations at other services. It’s going to be a fun couple of years.

Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.