If Search Engines Could Read Your Mind

 

What if a search engine knew exactly what you were thinking and unerringly provided perfect search results? The idea isn’t as farfetched as it sounds.

When people complain about poor-quality or irrelevant search results, they almost never blame their own poorly formed request. Yet bad queries are a huge part of the problem. It’s remarkable search engines can take a two- or three-word query and make sense out of it. Lacking context, search engines are forced to virtually guess at your true intent.

If only we searchers would spell out our needs in greater detail. Then, search engines wouldn’t have to rely so heavily on unreliable heuristics and imperfect meaning indicators. Our laxity is one reason most search engines offer search refinement tools, suggesting additional query terms, clustering results into conceptually related topics, or otherwise trying to extract more clues about what a searcher is really looking for.

These refinement tools are relatively primitive, but for good reason. Most of us are too lazy to take advantage of them; when we do, our results often improve. More often, we’re in too big a hurry to get results. We’d much rather waste time scanning results and clicking back and forth between less-than-useful pages than craft a really good query or use search refinement tools.

As we do this, the search engines are observing and learning from our fumbling activities. All the major search engines employ AI (define) experts who quietly build common sense and worldly knowledge into our search tools.

Wait — wasn’t AI widely discredited, after promising miracles in the ’80s? Some of the grander claims made by AI proponents at the time have indeed fallen by the wayside. But AI is alive and well. There have been serious breakthroughs, too. Computerized voice recognition systems are a significant example of applied AI.

20Q.net, an online version of Twenty Questions, is an example of what AI could do for search. The premise behind the site is simple: Think of a common object, then answer a series of questions. 20Q will then “guess” what you’re thinking about. Just about every time I’ve used it, 20Q has correctly identified the object I’ve imagined with fewer than 20 questions.

If 20Q can’t guess what you’re thinking, you “win” the game. The system then presents you with a list of other possible objects you might have been thinking about. If the object is in the list, click on it. 20Q will explain its logic to you, indicating contradictions between your answers and its own knowledge base. These aren’t necessarily “wrong,” rather an indicator of the learning 20Q has gained by interacting with other people.

20Q is a neural network that works much like the human brain. The software has been “trained” by thousands of users playing the game over the past decade. By interacting with users, the neural net has learned about real-world objects and continues to learn as it analyzes each game.

20Q has about 10 million synaptic connections (by contrast, the human brain has about 100 trillion). The game uses the neural network to choose each question it asks you. After collecting enough clues, the neural net guesses what you’re thinking about.

To a certain degree, search engines already employ similar systems. Just as 20Q starts out with broad questions (is it classified as Animal, Vegetable, or Mineral?) to prune the tree of possible branches, search engines do the same with the few clues offered by search terms, eliminating thousands or millions of possibilities before considering possible matches.

These learning systems will improve, enabling the search engines to offer better results even if we searchers continue with our miser-ish search habits. In the words of Google’s former VP of engineering, Wayne Rosing, “The day will come when Google won’t be a search engine anymore, because everything will be searchable. So, instead, we’ll have to algorithmically find you the good stuff. It will be an up-leveling of our ranking function, if you will, from what’s the best document to what’s the best, most well-formed knowledge on the subject.”

For more information about 20Q, see its FAQ page or take a quick tour of the game.

Join us for Search Engine Strategies August 8-11 in San Jose, CA.

Want more search information? ClickZ SEM Archives contain all our search columns, organized by topic.