Text Mining: MI5's (and Your) Secret Weapon

  |  February 22, 2005   |  Comments

If your customers are talking to you, shouldn't you listen to what they're saying? Text mining can help.

Years ago, I received my first large data file. It contained around 200,000 records. All had the names, addresses, and purchase histories of a retail catalog company's customers. Soon after I received the file, a question arose regarding how the company was marketing some of its products.

Being young and naive, I thought it would be no problem to use that data file to call appropriate customers and ask them questions about the company's marketing. After all, the file contained customers' phone numbers. I figured, "Who better to answer the questions?"

The customers were surprised by the calls but happy to answer my questions. The information we collected was helpful to the company. Fortunately, I didn't get in trouble for using the data in this manner.

Consumer Feedback

Customers are more than happy to give you feedback regarding your products and services. Some companies' customers actually submit unsolicited feedback. The companies could use the data to improve their products and services, yet often they don't.

Consumer packaged good, hotel, and airline companies are examples of organizations that receive unsolicited feedback all the time, usually in the form of complaint letters. Airlines receive thousands upon thousands of letters each year. What do they do with them?

Almost nothing.

Do they really take the time to read all the letters they receive? Of course not. Is there useful information contained somewhere in all of those letters? Of course there is!

Text Mining

It's probably not cost-effective to individually read and monitor all consumer complaint letters at companies that receive very large quantities. Fortunately, that's not necessary.

Consumer data can be analyzed with software known as text mining applications. The leader in this field is Autonomy. The algorithms underlying the software were first put into use by MI5, the U.K.'s Security Service, in defense applications. Today, they're available to any company interested in using them to mine available text data.

SPSS offers another text mining program. SPSS focuses on practical applications, so the software has a specialized module for analyzing text captured in surveys. SAS also offers a text mining product.

How It Works

Technical details of how text mining works involve mathematical discussions of things such as vectors, cosines between vectors, and cluster analysis. Thankfully, all these things are programmed into the software, allowing users to focus on the information generated from the analysis.

Here's a very simplified description: Text mining involves counting the number of times specific words appear in documents and comparing documents against one another in terms of word count. This helps group similar documents together. In an analysis of airline complaint letters, resultant groups might include "weather incidents," "baggage handling," and "flight attendant compliments."

Initial and Ongoing Use

The initial application of this kind of software helps a company learn what natural groupings of complaint letters exist in its data. By themselves, these results could be very useful.

On an ongoing basis, this kind of analysis could keep track of the quantity of letters received in each of those categories and determine when a new category appears (e.g., "leg room problems"). If the company applied it in a more in-depth manner, it could monitor the number of letters regarding a specific product or problem (e.g., "flight 435 out of MCI airport").

Works for Them; Works for You

The MI5's original techniques monitored information from many organizations and determined if "chatter" in certain categories was increasing. The same techniques can be used to determine if complaints in certain categories are on the rise.

Many companies have large data repositories they could use to improve their products and services, whether with an initial text-mining analysis or an ongoing analysis. If your customers are talking to you, shouldn't you listen to what they're saying?

ClickZ Live New York What's New for 2015?
You spoke, we listened! ClickZ Live New York (Mar 30-Apr 1) is back with a brand new streamlined agenda. Don't miss the latest digital marketing tips, tricks and tools that will make you re-think your strategy and revolutionize your marketing campaigns. Super Saver Rates are available now. Register today!

ABOUT THE AUTHOR

Brian Teasley Brian Teasley is the leader of Teasley, a consultancy that helpsadvertisers, marketers and advertising agencies use data and analysis toimprove their marketing campaigns. Brian has over 14 years experience inengineering and marketing, and has worked for numerous Fortune 100companies. Brian also teaches a marketing course at New York University. Heholds a M.S. degree in Applied Statistics from Iowa State University and aBA in Mathematics and a BA in Mathematics and Statistics from St. OlafCollege.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

UPCOMING TRAINING

Featured White Papers

Google My Business Listings Demystified

Google My Business Listings Demystified
To help brands control how they appear online, Google has developed a new offering: Google My Business Locations. This whitepaper helps marketers understand how to use this powerful new tool.

5 Ways to Personalize Beyond the Subject Line

5 Ways to Personalize Beyond the Subject Line
82 percent of shoppers say they would buy more items from a brand if the emails they sent were more personalized. This white paper offer five tactics that will personalize your email beyond the subject line and drive real business growth.

WEBINARS

Resources

Jobs

    • Customer Service Consultant
      Customer Service Consultant (Bonner and Partners) - Delray BeachBonner & Partners: Full-time Customer Service Consultant Position Who we are...
    • Financial Editor
      Financial Editor (Confidential) - DurhamSIX FIGURE EDITORS WANTED: To enforce lofty NEW editing standards. Easy Conditions Unlikely. Promotion and...
    • Information Processing Specialist
      Information Processing Specialist (Agora Inc. ) - BaltimoreInformation Processing Specialist – The IP specialist position ensures the successful...