Consilience - The Intrinsic Value of Big Data

  |  July 5, 2012   |  Comments

Big data has arrived due to volume, variety, and velocity. Get used to it.

When I first saw the Internet in 1993, I was dumbstruck by the obvious wonderfulness of its potential.

I had been in sales and marketing for 15 years and was knowledgeable about networks. I also understood humans' overwhelming need to communicate and figured this was sure to be, well, wonderful.

I felt the same way about big data as soon as I heard IBM's Jeff Jonas present at the eMetrics Marketing Optimization Summit last October in New York. But I've been much less able to explain why until I added a new word to my vocabulary.

Big Data Defined

In common parlance, big data has arrived due to:

  1. Volume. We have more data than ever imagined.
  2. Variety. Who knew we would be tracking and collecting mouse movements on websites or tire pressure and temperature?
  3. Velocity. Near real-time monitoring is essential when operating in a social media world.

In a nutshell, big data is more data than we are accustomed to. Get used to not being able to get used to how much data you have. It'll never stop.

1950-s-computer

Big Data Promise

The breakthrough was a technical one but the popular consensus is that the more data you have, the more knowledge you can glean from it. It's a bit like the old joke about the optimistic boy and the room full of horse manure…

(Side bar) Worried that their son was too optimistic, the parents of a little boy took him to a psychiatrist. Trying to dampen the boy's spirits, the psychiatrist showed him into a room piled high with nothing but horse manure. Yet instead of displaying distaste, the little boy clambered to the top of the pile and began digging.

"What do you think you're doing?" the psychiatrist asked.

"With all this manure," the little boy replied, beaming, "there must be a pony in here somewhere!" (/side bar)

Big Data Broken Promise

While this distributed processing technology lets us tackle larger and larger datasets, it doesn't do a thing to help us with analysis.

We're still digging through the same old manure in the same old way - but with a bigger shovel.

bulldozer-detail

Big Data Hope

Jeff Jonas' claim is that we can build systems where "data finds data."

The idea, and the word for it, is consilience.

Merriam Webster defines consilience as, "the linking together of principles from different disciplines especially when forming a comprehensive theory."

Wikipedia says:

"In science and history, consilience (also convergence of evidence or concordance of evidence) refers to the principle that evidence from independent, unrelated sources can 'converge' to strong conclusions. That is, when multiple sources of evidence are in agreement, the conclusion can be very strong even when none of the individual sources of evidence are very strong on their own."

In chapter 7 of the book "Beautiful Data: The Stories Behind Elegant Data Solutions," Jeff Jonas wrote:

"Next-generation 'Smart' information management systems will not rely on users dreaming up smart questions to ask computers; rather, they will automatically determine if new observations reveal something of sufficient interest to warrant some reaction, e.g., sending an automatic notification to a user or a system about an opportunity or risk.

... When the 'data can find the data,' there exists an opportunity for the insight to find the user."

So I still live in hope that big data, beyond being the latest catch phrase for all marketing mavens ("Spicy Chicken McBites - Now with BIG DATA!"), will evolve from merely being able to manage large datasets to systems that can participate in that uniquely human act of discovery.

The day will come when the system pops up an alert that says, "Gee, that's funny..."

insight

1950's Computer, Bulldozer Detail, and Insight images from Shutterstock.

Tags:

ClickZ Live Chicago Join the Industry's Leading eCommerce & Direct Marketing Experts in Chicago
ClickZ Live Chicago (Nov 3-6) will deliver over 50 sessions across 4 days and 10 individual tracks, including Data-Driven Marketing, Social, Mobile, Display, Search and Email. Check out the full agenda and register by Friday, Oct 3 to take advantage of Early Bird Rates!

ABOUT THE AUTHOR

Jim Sterne

Jim Sterne is an international consultant who focuses on measuring the value of the Web as a medium for creating and strengthening customer relationships. Sterne has written eight books on using the Internet for marketing, is the founding president and current chairman of the Digital Analytics Association and produces the eMetrics Summit and the Media Analytics Summit.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!

COMMENTS

UPCOMING EVENTS

Featured White Papers

IBM: Social Analytics - The Science Behind Social Media Marketing

IBM Social Analytics: The Science Behind Social Media Marketing
80% of internet users say they prefer to connect with brands via Facebook. 65% of social media users say they use it to learn more about brands, products and services. Learn about how to find more about customers' attitudes, preferences and buying habits from what they say on social media channels.

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising

An Introduction to Marketing Attribution: Selecting the Right Model for Search, Display & Social Advertising
If you're considering implementing a marketing attribution model to measure and optimize your programs, this paper is a great introduction. It also includes real-life tips from marketers who have successfully implemented attribution in their organizations.

Jobs

    • Internet Marketing Campaign Manager
      Internet Marketing Campaign Manager (Straight North, LLC) - Downers GroveWe are looking for a talented Internet Marketing Campaign Manager...
    • Internet Marketing Specialist
      Internet Marketing Specialist (InteractRV) - DallasInternet Marketing Specialist InteractRV - Anywhere Telecommute, USA SEM | SEO | Content Creator...
    • Tier 1 Support Specialist
      Tier 1 Support Specialist (Agora Inc.) - BaltimoreThis position requires a highly motivated and multifaceted individual to contribute to and be...