Alexa Opens Its Search Index’s Alexa unit is opening up its Web crawl infrastructure to developers, paving the way for the creation of low-cost niche search engines and applications enabled by Alexa’s vast Web traffic data.

The Alexa Web Search Platform beta was released yesterday. It basically allows developers to tap into Alexa’s index of 4 billion Web pages, search against a selected set of data, and to create their own vertical search engine.

Developers will have the chance to boost their search and application development efforts with Alexa’s crawl, storage, processing, search and server technologies. Pricing will be based on usage, with fees for processing, storage and bandwidth used.

“Just imagine all the talented entrepreneurs who have been stymied by a lack of Web-scale tools and data. Now, for less than the cost of an iPod, they can get into the search field and begin inventing and creating,” said Geoffrey Mack, product manager at Alexa, in the Alexa blog. “You as a consumer can begin using these new search services, and all of us can begin reaping the benefits of an expanded search space with hundreds of wholly new search services being created by anybody with an idea and a credit card.”

The idea is not new, having been tried unsuccessfully by several companies in 2000, Danny Sullivan, editor of Search Engine Watch, notes in his blog. Those services failed with the dot-com bust, because there was no market for vertical search or search ads.

That much has changed. Search engines are finding users and advertisers alike are seeking more targeted vertical offerings. Sullivan points to a handful of providers offering similar services for free, including Rollyo and Gigablast. Similar results can also be achieved by developers using the APIs from Google and Yahoo, he adds.

While Sullivan sees the move as mildly positive in its potential to spur the market leaders into innovation, he notes that the rise of search ads have made the idea of leasing search infrastructure an obsolete business model, at least for larger players.

“The current fight for AOL between MSN and Google underscores this,” Sullivan writes. “AOL isn’t being asked to stump up money for Web search to those companies. It has an audience that MSN and Google want to reach. They stumble over themselves to see who can offer the best deal.”

Others see the move as more groundbreaking, with the potential to commoditize a search engine’s index and shifting its value to the engine’s relevance ranking and other services offered to users.

“This furthers the notion that the value of a search engine is in the application(s) built on top of it, like ad networks and the ability to match relevant ads to content, or any specific vertical search functionality,” Jeff Clavier, managing partner at SoftTech Venture Consulting, and co-chair of the SDForum Search Special Interest Group, a Silicon Valley non-profit group, wrote in his blog.

Related reading