Don't Let Big Data Play Games With Your Brain

  |  June 5, 2014   |  Comments

The next time you see a pattern, a connection, or a correlation, remember that you may not be the best person to determine if it passes the smell test. You will do your best work in collaboration with a proper subject matter expert.

In my last article ("10 Things Not to Do as a Digital Analyst"), I discussed a word I had just learned: iatrogenesis, the "inadvertent and preventable induction of disease or complications by the medical treatment or procedures of a physician or surgeon."

games-with-your-brain1This time, I'm upping the ante to two words: pareidolia and apophenia.

If you are more than 50 years old, you actually experienced snow on TV rather than just seeing it in reruns of Poltergeist. Visual static, that fuzzy, random, all-noise/no-signal gray blur was hypnotically compelling because you could swear you saw things in it.

(Image source)

While a small percentage of those who were young in the ‘60s legitimately saw things due to illegitimate hallucinogenics, the rest saw things because of pareidolia.

According to Wikipedia, pareidolia is "a psychological phenomenon involving a vague and random stimulus (often an image or sound) being perceived as significant."

games-with-your-brain2That stranger outside in the dark that is a bush and a clothes line in the morning? Pareidolia. That monster in the closet that resolves into a jacket and a pile of dirty clothes by the light of day? Pareidolia. Jesus on grilled cheese? Pareidolia.

We are trained at a very young age to actively exercise our pareidolia. We are encouraged to lie back in the grass and look up at the clouds and are emotionally rewarded when we declare this one a shark and that one Mongolia.

We happily wiggle this loose cognitive tooth by playing pareidoliac tricks on ourselves with optical illusions. Is it a duck or a rabbit? A young maid or a crone? A vase or two faces? And which direction is that dancer turning, anyway?

When pareidolia gets more sophisticated, it goes beyond the visual.

The gambler's fallacy is another example. Get heads 99 times in a row when flipping a coin and you would be mocked as a fool for not betting that the next one is going to be tails. It just has to be. The odds are overwhelming! In fact, the odds are still 49.999 percent that the next flip will, again, be heads. Yes, I have personally seen a flipped quarter bounce and come to a rest on its side.

When the little metal ball falls into another black slot on the roulette wheel 20 times in a row it has no statistical impact on whether it will come up red or black next time. But the gambler's fallacy insists on betting on red.

At this point, we leave the world of pareidolia behind and head straight into the arms of conspiracy theorists.

Apophenia is, "the experience of seeing patterns or connections in random or meaningless data."

Evolution has ensured that we make these connections by killing off those who ignore them. If you hear a deep growl in the underbrush, it's much safer to assume you are about to be eaten and be wrong than to assume you will not and be dinner. Brains that see patterns and connections survived more often in order to have offspring.

This is the statistician's Type I error. Uncharitably called jumping to conclusions, the false positive is a common problem. But not as dangerous as Type II, not seeing the connection between hygiene and health (John Snow, mapping out cholera in 1854 London; Ignaz Semmelweis working out the correlation of hand-washing to infant mortality about the same time in Vienna.). Finding patterns is better than not finding patterns.

We are so dedicated to finding patterns that we have spent a great deal of time, trouble, and thought on creating complex algorithms running on multitudes of meshed machines to do just that. We taught programs to teach themselves how to find patterns. And then, we let them loose on Big Data.

Machine-learning programs connect the dots for us, showing us relationships that are unexpected, fascinating, and potentially game-changing.

But we humans must be hyper-critical of our silicon slaves. We are easily fooled between correlation and causation because the human brain wants the puzzle pieces to fall neatly into place. It wants to believe.

The next time you see a pattern, a connection, or a correlation, remember that you may not be the best person to determine if it passes the smell test. You will do your best work in collaboration with a proper subject matter expert.

cloud-mind-gamesYes, you should write the best queries you can.

Yes, you should use the most massive Hadoop clusters you can get your hands on.

Yes, you should slice and dice until a pattern emerges.

This is your job.

But your responsibility is to take your findings to a subject matter expert who knows the real world. It's not up to them to admire the genius of your elegant mathematical model, but to see if they can see a shark, the outline of Mongolia or just a cloud. Don't let Big Data play games with your brain.


Jim Sterne

Jim Sterne is an international consultant who focuses on measuring the value of the Web as a medium for creating and strengthening customer relationships. Sterne has written eight books on using the Internet for marketing, is the founding president and current chairman of the Digital Analytics Association and produces the eMetrics Summit and the Media Analytics Summit.

COMMENTSCommenting policy

comments powered by Disqus

Get the ClickZ Analytics newsletter delivered to you. Subscribe today!



Featured White Papers

US Consumer Device Preference Report

US Consumer Device Preference Report
Traditionally desktops have shown to convert better than mobile devices however, 2015 might be a tipping point for mobile conversions! Download this report to find why mobile users are more important then ever.

E-Commerce Customer Lifecycle

E-Commerce Customer Lifecycle
Have you ever wondered what factors influence online spending or why shoppers abandon their cart? This data-rich infogram offers actionable insight into creating a more seamless online shopping experience across the multiple devices consumers are using.




  • SEO Specialist
    SEO Specialist (Marcel Digital) - ChicagoSearch Engine Optimization (SEO) Specialist   Marcel Digital is an award winning digital marketing...
  • SEO / SEM Manager
    SEO / SEM Manager (CustomInk) - FairfaxAre you a friendly, motivated, and inquisitive individual? Are you a positive, out-going leader? Are you...
  • SEO Analyst
    SEO Analyst (XO Group) - New YorkSEO Analyst @ XO Group About this Job, You and Our Team: The XO Group SEO Team is looking for you, a passionate...