Google Unveils Sitemaps Tool
The program provides Web site owners with more control over how their pages are indexed.
Google unveiled a new program today which would provide Web site owners and marketers more control over when and how their pages are indexed by the search engine’s crawlers.
“We’re undertaking an experiment called Google Sitemaps that will either fail miserably, or succeed beyond our wildest dreams, in making the Web better for Webmasters and users alike,” said Shiva Shivakumar, engineering director at Google, in a post on Google’s blog.
The Sitemaps program aims to optimize Google’s crawling activities, leading to better search results, and to give site owners more input into how their sites are crawled. The process involves a site owner creating and posting an XML file on the site’s server describing which pages on the site should be indexed, when those pages were last updated, how often they’re updated, and how important each page is relative to others on the site. Google created an open-source tool, Sitemap Generator, to assist in the process.
This potentially gives marketers a better shot at having important site pages indexed more frequently. It will particularly benefit sites built in a way that makes Google’s traditional crawling ineffective, said Matt Naeger, VP and general counsel of SEM firm Impaqt.
“A surprising amount of sites have problems getting indexed. There are ways to improve that with a proper redesign, and with techniques like a robots file. This is a potential shortcut to that process,” Naeger said.
One way marketers in particular may benefit from the program would be to quickly index pages for a time-sensitive campaign or promotion. Many marketers forego organic SEO for such campaigns and instead focus on pay-per-click search ads, which deliver faster results, he observed.
“Initially, we plan to use the URL information Webmasters supply to further improve the coverage and freshness of our index. Over time that will lead to our doing an even better job of delivering more search results from more Websites,” said Shivakumar.
The program doesn’t replace Google’s regular crawling activities, but rather complements them. Google doesn’t guarantee all sites that submit a Sitemap, or all pages included on the Sitemap, will be crawled and indexed.
The move is also a strike against competitors, boosting the size of Google’s index in the ongoing battle for search engine dominance against Yahoo and MSN. It also provides Google with an answer to Yahoo’s paid inclusion program, without the “paid” part, while also countering MSN’s promise to index sites more frequently than Google.
