Text Mining: MI5's (and Your) Secret Weapon

If your customers are talking to you, shouldn't you listen to what they're saying? Text mining can help.

Years ago, I received my first large data file. It contained around 200,000 records. All had the names, addresses, and purchase histories of a retail catalog company’s customers. Soon after I received the file, a question arose regarding how the company was marketing some of its products.

Being young and naive, I thought it would be no problem to use that data file to call appropriate customers and ask them questions about the company’s marketing. After all, the file contained customers’ phone numbers. I figured, “Who better to answer the questions?”

The customers were surprised by the calls but happy to answer my questions. The information we collected was helpful to the company. Fortunately, I didn’t get in trouble for using the data in this manner.

Consumer Feedback

Customers are more than happy to give you feedback regarding your products and services. Some companies’ customers actually submit unsolicited feedback. The companies could use the data to improve their products and services, yet often they don’t.

Consumer packaged good, hotel, and airline companies are examples of organizations that receive unsolicited feedback all the time, usually in the form of complaint letters. Airlines receive thousands upon thousands of letters each year. What do they do with them?

Almost nothing.

Do they really take the time to read all the letters they receive? Of course not. Is there useful information contained somewhere in all of those letters? Of course there is!

Text Mining

It’s probably not cost-effective to individually read and monitor all consumer complaint letters at companies that receive very large quantities. Fortunately, that’s not necessary.

Consumer data can be analyzed with software known as text mining applications. The leader in this field is Autonomy. The algorithms underlying the software were first put into use by MI5, the U.K.’s Security Service, in defense applications. Today, they’re available to any company interested in using them to mine available text data.

SPSS offers another text mining program. SPSS focuses on practical applications, so the software has a specialized module for analyzing text captured in surveys. SAS also offers a text mining product.

How It Works

Technical details of how text mining works involve mathematical discussions of things such as vectors, cosines between vectors, and cluster analysis. Thankfully, all these things are programmed into the software, allowing users to focus on the information generated from the analysis.

Here’s a very simplified description: Text mining involves counting the number of times specific words appear in documents and comparing documents against one another in terms of word count. This helps group similar documents together. In an analysis of airline complaint letters, resultant groups might include “weather incidents,” “baggage handling,” and “flight attendant compliments.”

Initial and Ongoing Use

The initial application of this kind of software helps a company learn what natural groupings of complaint letters exist in its data. By themselves, these results could be very useful.

On an ongoing basis, this kind of analysis could keep track of the quantity of letters received in each of those categories and determine when a new category appears (e.g., “leg room problems”). If the company applied it in a more in-depth manner, it could monitor the number of letters regarding a specific product or problem (e.g., “flight 435 out of MCI airport”).

Works for Them; Works for You

The MI5’s original techniques monitored information from many organizations and determined if “chatter” in certain categories was increasing. The same techniques can be used to determine if complaints in certain categories are on the rise.

Many companies have large data repositories they could use to improve their products and services, whether with an initial text-mining analysis or an ongoing analysis. If your customers are talking to you, shouldn’t you listen to what they’re saying?

Subscribe to get your daily business insights

Whitepapers

US Mobile Streaming Behavior
Whitepaper | Mobile

US Mobile Streaming Behavior

5y

US Mobile Streaming Behavior

Streaming has become a staple of US media-viewing habits. Streaming video, however, still comes with a variety of pesky frustrations that viewers are ...

View resource
Winning the Data Game: Digital Analytics Tactics for Media Groups
Whitepaper | Analyzing Customer Data

Winning the Data Game: Digital Analytics Tactics for Media Groups

5y

Winning the Data Game: Digital Analytics Tactics f...

Data is the lifeblood of so many companies today. You need more of it, all of which at higher quality, and all the meanwhile being compliant with data...

View resource
Learning to win the talent war: how digital marketing can develop its people
Whitepaper | Digital Marketing

Learning to win the talent war: how digital marketing can develop its peopl...

2y

Learning to win the talent war: how digital market...

This report documents the findings of a Fireside chat held by ClickZ in the first quarter of 2022. It provides expert insight on how companies can ret...

View resource
Engagement To Empowerment - Winning in Today's Experience Economy
Report | Digital Transformation

Engagement To Empowerment - Winning in Today's Experience Economy

1m

Engagement To Empowerment - Winning in Today's Exp...

Customers decide fast, influenced by only 2.5 touchpoints – globally! Make sure your brand shines in those critical moments. Read More...

View resource