Finding Disaster Coverage At Search Engines

Following the unprecedented terrorist attacks on the United States last week, Web users turned en masse to search engines for information. It took those services some time to adjust to the demand, but as the day progressed many came up to speed.

AltaVista Immediately Relevant

At first, the shining star among the major services was AltaVista. This is because AltaVista has a partnership with news crawler Moreover and integrates Moreover’s news headlines right into its regular listings.

As a result, a search for “World Trade Center” on AltaVista less than two hours after the event, if not earlier, brought up news stories about the tragedy.

In contrast, Google was a great disappointment. Google also integrates news headlines into its search results, but its news coverage is generally far less comprehensive or reliable than AltaVista’s. Two hours after the attack, a search for “World Trade Center” showed no indication in Google’s results that a mass disaster had happened.

Nor did other search engines prove more useful immediately after the attacks. Searches for “World Trade Center” at MSN Search, Ask Jeeves, and Yahoo, for example, brought back no listings relevant to the current catastrophe. Nor did selecting special news links, when available, from their home pages.

Part of this was no doubt the result of many major search engines being based on the West Coast of the United States. When the attacks occurred in New York and Washington, people weren’t even in their offices on the West Coast.

Finally Reacting to Disaster

As the day moved on, the search engines started getting their act together. About four hours after the attack, Google placed on its home page links to news coverage, including a link to a page copied directly from CNN.

Though probably violating CNN’s copyright, Google will no doubt be forgiven, given that many Web users were unable to access the CNN site itself to get coverage. The cached copy of the page was fully attributed to CNN, along with links to CNN and a note that the copy was provided as a public service.

Despite the addition of links to the Google home page, nothing changed in Google’s search results. That was a problem, because many users gravitate to a search engine’s search box and ignore relevant information on the search engine’s home page. In other words, if you don’t talk to them within your search results, you aren’t talking to them at all.

To resolve this issue, Google changed again about six hours after the attacks, adding a special link that appeared in response to a search for “World Trade Center.”

That special link came at the top of Google’s results, where an ad might ordinarily reside. Indeed, the link even said “Sponsored Link” next to it, although this was almost certainly because Google couldn’t quickly remove the standard wording for links that appeared in that ad area.

Clicking on the link brought users to a special “Current Events” page, which listed major news sources and cached articles from major news sources, as well as the home pages of United Airlines and American Airlines, both of which were difficult or impossible for many Web users to reach.

About four hours after the attacks, other search services also began making significant alterations. AltaVista was already ahead of its competitors in providing relevant links right in its results, but the service added a special “news” search box right at the top of its home page to better direct users to its news-specific results.

By six hours after the attack, the majority of AltaVista’s above-the-fold area of the home page was devoted to the attacks, along with a phone number on how people could donate blood.

At Yahoo, about four hours after the disaster, the “mantle” box was changed to put information about the attacks on the home page. That box normally promotes such things as shopping and auctions. It was replaced so that, as at AltaVista, the disaster was the dominant page element.

In contrast, coverage at Lycos was harder to spot during the first few hours after the attack. There was no above-the-fold positioning. Instead, the site looked fairly normal, though it did contain several links to coverage.

Over at MSN, the situation was the same. Four hours after the attack, the front page had been altered — and more substantially than at Lycos — but not to the extent of the front pages at Yahoo and AltaVista.

Fixing the Results

How about the all-important results pages? At Yahoo, the first major link led to its news coverage, while the remainder of the page stayed “normal” — that is, no results were relevant to the disaster.

Over at Lycos, the results hadn’t changed at all. In response to a search on “World Trade Center,” absolutely nothing indicated that a terrorist attack had occurred.

Ask Jeeves, which should have been right on top of this type of search need, was no better. The search engine made its name by having human editors review and compile relevant links to come up for top searches. Four hours after the attacks, no such links were evident. Nor had the home page been altered in any way.

Things changed at Ask Jeeves about six hours after the attack. A new link had been added just under the search box on the home page: “Get the latest news on America Under Seige [sic].” In turn, this brought up a special results page. Similar relevant links also began appearing in response to a search for “World Trade Center.”

An Ask Jeeves spokesperson also said that all travel ads — deemed inappropriate, given the hijackings — were removed.

Inktomi, which provides the main results that appear at iWon and other search engines, said it “stuffed” some queries by the fourth hour after the attack so that at least one relevant link would appear in response to searches about the attack.

For example, a search for “World Trade Center” was programmed to list CNN first, with a special description that said, “CNN is following the terrorist attack on the World Trade Center and the Pentagon.” Without this stuffed link, results at a place such as iWon would have been embarrassingly absent in acknowledgment of the disaster.

As it was, some people undoubtedly found it odd that the third site listed in the results offered a live view from the top of the World Trade Center. This is simply an example of how a search engine needs time to refresh its database with new information. A similar link could also be found at Google, so iWon’s Inktomi results are not unique with this problem. In addition, it also takes time for the Web sites themselves to make updates. Until those are done, there’s no easy way for the crawler-based search engines to get their own listings updated.

At Netscape Search, a spokesperson said its results were going to be changed so that the “Netscape Recommends” section would have relevant results for queries related to the attacks. However, these hadn’t been added when the site was checked eight hours after the attacks.

Sibling service AOL Search had such links up at least six hours after the attack and said the links actually were added within minutes of it happening. AOL also said many people in its chat areas were seeking out and passing along information.

Heavy Use and Search Requests

Traffic to search engines and portals was up substantially, up to 10 times the normal amount, according to representatives.

“We have seen traffic to the home page, and specifically Lycos News and Wired News, increase tenfold since the onset of this morning’s tragedy,” said Lycos spokesperson Kathy O’Reilly last Tuesday.

At MSN Search, traffic was said to be twice normal.

What were people searching for? News sites with coverage of the attacks or direct links to coverage itself.

For example, Lycos estimated that the term “CNN” received 160 times as many searches during the 3-hour period immediately following the attacks as it normally gets during an average 12-hour period.

Also at Lycos, the phrase “World Trade Center” received over 50 times as many searches as last week’s third-most popular term, “NFL.”

From midnight through noon Eastern Daylight Time on September 11, only 1 of the top 20 most-popular search terms on Lycos wasn’t related to the disaster: “Whitney Houston” (she was a no-show at the Michael Jackson concert on September 10).

Editor’s note: A longer version of this article, including screen shots of the various searches described, resides here at sister publication Search Engine Watch.

Related reading