Accuracy Audits in the Real World

Most people would say sending a dozen roses to a friend on his birthday is a good thing. But like many good things, sending flowers can go entirely wrong if bad data gets in the way. Bad data can lead to very strange situations — even in your personal life.

An Upgraded Contact Database

A year ago, many of us began receiving lots of e-mail from people they hadn’t heard from in a long time. The culprit was Plaxo. The service looks into your Outlook contacts and sends e-mails to everyone there so they can update their contact information. Like most people, I simply put these messages in the bottom of my “follow up” folder and don’t give them another thought.

When I finally had a lazy afternoon, I decided to update my contact information using everything I’d received from Plaxo. I was struck by the diversity of people who’d e-mailed me. Some of them were sales prospects I’d met only once. Some were Fortune 500 executives, some were college friends who still call me by the nickname I guess I deserved during freshman year. In any case, I filled out the updates, then signed up for the service myself.

During the signup process, the service asked me if I wanted to send an e-mail to all of my contacts in MS Outlook requesting they update their contact information. I’d never before sent an e-mail to everyone in my database, totaling over 1,500 people. I probably should have kept it that way.

I was shocked to see the conversion percentage of people who responded to my request. Over 50 percent responded within the first 3 days. After further analysis and talking to people, I came to the conclusion that egotism, of all things, was driving the high conversion rate. People wanted to make sure I knew how to spell their name properly, my database had their new (usually better) job title, A more upscale home address, and so on. It was an interesting lesson in human motivation.

In any case, I dutifully updated everyone’s contact information and thought nothing more about it.

A Practical Joke Gone Wrong

An more amusing part of this story took place shortly after. I received an Outlook reminder for Joe Smith’s (not his real name) birthday. It turned out I’d met Joe at a Web analytics conference. Why was his birthday in my calendar? I forgot about it, but a week later, the same thing happened again. Another reminder, about another vague acquaintance’s birthday.

I soon realized these people had included their birthday when they updated their contact information in response to my request. This went on for about 6 months. As I learned more about my acquaintances’ birthdays I began to feel a bit voyeuristic. Occasionally, I’d ask someone if they had a good birthday. I’m sure they wondered how I knew.

Then two weeks ago, I received another birthday reminder for a friend I’ve played basketball with twice a week for the 8 last years. We routinely e-mail a group of guys to confirm we’ll be getting up early the next day to play. For the sheer heck of it, I decided to send him a dozen birthday roses. (In case you’re wondering if I had other motivations, my wife just delivered our second baby.)

I ordered delivery the next day. It took a whole day before I heard back from him, during which time I wondered if he thought I was crazy. He finally reached my voice-mail with a bewildered message. He thanked me for the roses, but explained he was confused: his birthday was 6 months earlier.

After looking some more into Outlook, we realized his contact file had a biannual birthday reminder.

Data Accuracy is a Serious Problem

I’d just had an adventure in data inaccuracy. Many companies and their Web analysts suffer from this problem. As a result, they make bad decisions, like my decision to send flowers 6 months after my friend’s birthday. (OK, I realize that may not have been my only bad decision in the incident). In any case, the very first and most important step in any Web analytics engagement is to ensure the data you’re getting is accurate so that any decisions based on it are sound.

Common problems include:

• Tracking not based on an accurate, persistent cookie.
• Internal visitors and traffic incorrectly added to totals.
• Tracking spiders and robots treated as real visitors.
• Missing pages served by caching servers.
• Inaccurate filters set up within the tracking tool.

For more examples of how to make your online data more accurate, view my colleague Jason Burby’s articles here and hereHave any fun adventures of your own in data accuracy? E-mail me. I may include them in a future column.