Don’t Let Big Data Play Games With Your Brain

In my last article (“10 Things Not to Do as a Digital Analyst“), I discussed a word I had just learned: iatrogenesis, the “inadvertent and preventable induction of disease or complications by the medical treatment or procedures of a physician or surgeon.”

games-with-your-brain1This time, I’m upping the ante to two words: pareidolia and apophenia.

If you are more than 50 years old, you actually experienced snow on TV rather than just seeing it in reruns of Poltergeist. Visual static, that fuzzy, random, all-noise/no-signal gray blur was hypnotically compelling because you could swear you saw things in it.

(Image source)

While a small percentage of those who were young in the ‘60s legitimately saw things due to illegitimate hallucinogenics, the rest saw things because of pareidolia.

According to Wikipedia, pareidolia is “a psychological phenomenon involving a vague and random stimulus (often an image or sound) being perceived as significant.”

games-with-your-brain2That stranger outside in the dark that is a bush and a clothes line in the morning? Pareidolia. That monster in the closet that resolves into a jacket and a pile of dirty clothes by the light of day? Pareidolia. Jesus on grilled cheese? Pareidolia.

We are trained at a very young age to actively exercise our pareidolia. We are encouraged to lie back in the grass and look up at the clouds and are emotionally rewarded when we declare this one a shark and that one Mongolia.

We happily wiggle this loose cognitive tooth by playing pareidoliac tricks on ourselves with optical illusions. Is it a duck or a rabbit? A young maid or a crone? A vase or two faces? And which direction is that dancer turning, anyway?

When pareidolia gets more sophisticated, it goes beyond the visual.

The gambler’s fallacy is another example. Get heads 99 times in a row when flipping a coin and you would be mocked as a fool for not betting that the next one is going to be tails. It just has to be. The odds are overwhelming! In fact, the odds are still 49.999 percent that the next flip will, again, be heads. Yes, I have personally seen a flipped quarter bounce and come to a rest on its side.

When the little metal ball falls into another black slot on the roulette wheel 20 times in a row it has no statistical impact on whether it will come up red or black next time. But the gambler’s fallacy insists on betting on red.

At this point, we leave the world of pareidolia behind and head straight into the arms of conspiracy theorists.

Apophenia is, “the experience of seeing patterns or connections in random or meaningless data.”

Evolution has ensured that we make these connections by killing off those who ignore them. If you hear a deep growl in the underbrush, it’s much safer to assume you are about to be eaten and be wrong than to assume you will not and be dinner. Brains that see patterns and connections survived more often in order to have offspring.

This is the statistician’s Type I error. Uncharitably called jumping to conclusions, the false positive is a common problem. But not as dangerous as Type II, not seeing the connection between hygiene and health (John Snow, mapping out cholera in 1854 London; Ignaz Semmelweis working out the correlation of hand-washing to infant mortality about the same time in Vienna.). Finding patterns is better than not finding patterns.

We are so dedicated to finding patterns that we have spent a great deal of time, trouble, and thought on creating complex algorithms running on multitudes of meshed machines to do just that. We taught programs to teach themselves how to find patterns. And then, we let them loose on Big Data.

Machine-learning programs connect the dots for us, showing us relationships that are unexpected, fascinating, and potentially game-changing.

But we humans must be hyper-critical of our silicon slaves. We are easily fooled between correlation and causation because the human brain wants the puzzle pieces to fall neatly into place. It wants to believe.

The next time you see a pattern, a connection, or a correlation, remember that you may not be the best person to determine if it passes the smell test. You will do your best work in collaboration with a proper subject matter expert.

cloud-mind-gamesYes, you should write the best queries you can.

Yes, you should use the most massive Hadoop clusters you can get your hands on.

Yes, you should slice and dice until a pattern emerges.

This is your job.

But your responsibility is to take your findings to a subject matter expert who knows the real world. It’s not up to them to admire the genius of your elegant mathematical model, but to see if they can see a shark, the outline of Mongolia or just a cloud. Don’t let Big Data play games with your brain.

Related reading