Yahoo! Lab to Cook Up New Search Tech

Yahoo has created a research organization focusing on paid search and other technologies. It will be headed by the founder of Overture Research.

Yahoo Research Labs will help the company compete in the escalating fight for search superiority inspired by the explosive growth of the paid search sector, estimated to hit $6.3 billion in 2004, according to Jupiter Research, which shares a parent company with this publication.

With that kind of money involved, it’s no wonder companies are working hard to develop innovations. Yahoo competitor Google operates a similar lab, and even e-commerce player Amazon has started a subsidiary, A9, dedicated to search.

The engineers and scientists in Yahoo’s lab will study topics including paid search, Web search, vertical businesses and platform technologies. Gary William Flake, founder of Overture Research and former chief science officer of Overture Services, which is now owned by Yahoo, will head the lab.

“This is the second incarnation of Overture Research,” said Flake. Essentially, he said, Overture Research has been repurposed into Yahoo Research Labs. Overture Research, the brains behind Overture, the leading paid search entity and a pioneer in the field, has developed local search tools, spell correction, which helps users correct mistakes and matches queries to merchant results more effectively, and phrase- and broad-matching and content match tools over the last year.

Flake said he would be moving from Pasadena, Calif. to Sunnyvale, Calif. as he assumes his duties as head of the lab. The core research team of about 24 people will remain in Pasadena.

“Yahoo Research Labs will expand as time goes by,” said Flake. Also, Yahoo’s R&D people will work with the core research team.

Underscoring the importance of paid search, Yahoo credited its search and marketplace properties, along with other factors, for its soaring revenue and earnings in line with expectations in its fourth quarter earnings report last week.

Core research areas for the lab will include machine learning, statistical data mining and efficient algorithms.

The news comes as the industry prepares for Yahoo to make an algorithmic search switch from Google to Inktomi the next few months.

Asked if the lab will work with Inktomi after the switch, Flake said, “We have partnered and will continue to partner with the Web search unit of Inktomi for making improvements in Web search. We would not try to create a new type of search engine from scratch. We’ll be working with Inktomi on improving Web search not just incrementally but coming up with new algorithms that leapfrog the current state of the art.”

Because search engine marketing has taken off so explosively, online marketers and publishers are brainstorming ways to improve upon the category.

They are also aggressively hiring to beef up their own capacity for study in the area of Web search. In November, MSN hired Overture CTO Paul Ryan away from Overture to head its lab. Microsoft is working to build its own search technology and has been hiring aggressively.

Flake said the Yahoo Research Labs are not a reaction to MSN’s move.

“We had the seeds planted and the plans made for this decision well in advance of Paul’s departure,” Flake said.

Flake came to Yahoo when the company acquired Overture last year. Before that he was leader of NEC Research Institute’s Web data-mining program.

The scientist said he wants to avoid the “swinging pendulum” approach to research. With this approach, research can become either too focused on long-term goals to be practical or too focused on short-term “bottom line” goals to be creating long-term solutions to long-term problems.

“We value having a portfolio of activities that include long term, short term and medium-term activities,” Flake said. “Always having something in the pipeline that would deliver a month from now, six weeks from now, a year from now.”

“Yahoo has a history of being innovative. We think this is a step toward consolidating a lot of existing assets we have in the company and having a single point of contact for pursuing research topics that will have a high impact on the company moving forward.”