Big Data and Small Data: Where the Twain Meets

When is small beautiful in data analytics? That’s become a topic of energetic debate as talk about “big data” grows, while others promote “small data.”

As many parts of North America were digging out of ice and snow, I was fortunate to enjoy two sunny West Coast conferences that shed more light on the dispute. Tag management leaders Tealium and Ensighten each hosted user conferences for their customers and partners in the first weeks of the new year. These new technologies, of course, are about a great deal more than just tag management. They are core to tracking real-time, digital interactions with consumers across multiple channels. So what these conferences made clear is that next-gen tag management is a linchpin in the debate over “big data” and “small data.”

Allen Bonde of the Digital Clarity Group, who has taken the time to define terms in his Small Data Group Blog, believes small data may emerge as a “mainstream movement” this year – interesting, as investments in big data initiatives are growing like wildlife. The International Data Corporation (IDC) forecasts a big data market at $16.1 billion in 2014. According to the analyst firm, that’s six times faster than for IT overall.

So it may seem a bit contrarian to be talking about small data, but not actually. Small data in Allen Bonde’s definition relates to the “last mile of big data,” where it can be “organized and packaged,” made useable, and most importantly actionable in everyday work. It’s about marketing teams getting timely, meaningful insights out of data, rather than being overwhelmed by the volume and the velocity at which it accumulates.

But small data also refers to the digital traces we leave in our personal interactions, our jobs, and in the marketplace. Now, as the mobile revolution in the form of tablets, smartphones, smartwatches, and other wearable technologies delivers a new mountain of small data tracking the moments of our lives, we are just getting started with big opportunities for small data.

One of the most intriguing proposals for such data comes from Deborah Estrin, professor of computer science at Cornell, my alma mater. At TEDMED 2013, she called for service providers to repackage the data that we generate as individuals and return it to us. The data, she argues, can be used to track personal health and medical treatments. In effect, this would make data bi-directional. That’s potentially of great benefit in other applications as well if we can access our own digital breadcrumbs in useable formats. But for now let’s move onto marketing. Tag management, according to Tealium founder and chief executive (CEO) Ali Behnam, is really an “enabler for big data.” And, in the new nomenclature, we might say that tag management brings “big data” and “small data” together.

Digital Data and the Marketing Nirvana

I’ve argued for some time that marketing teams need less data, not more to support their decisions – but it has to be the right data. Big data is admittedly extremely important in tracking large-scale trends, making forecasts for large populations, and analyzing customer bases by a variety of critical factors. As a marketing team, you definitely want the analytical ability to segment large volumes of data about your customer base. But digital data about individuals, small data by definition, enables marketers to get to know real people and build 1:1 relationships – the marketing nirvana.

What’s critical is to have data (big or small) that’s actionable. (See my ClickZ column last July, Metrics That Matter: A New Framework to Reveal Truly Actionable Insights.) As Tealium’s Behnam argues, it comes down to “data activism,” made possible when companies invest in “systems, people, and processes to start taking action on the data and getting more value from their analytics.”

Next-Gen Tag Management

The good news is the new analytics and tag management technologies make it possible to centralize, analyze, and personalize digital data collection across the customer journey. Marketing teams can use these tools to access data through third-party tags and isolate specific digital data across the customer journey, without having to call on the technical folks in IT every time a new tag is required. That supports improved marketing agility and decisions in real time. Even more to the point is what Ensighten founder and CEO Josh Manion says. Tag management means marketing can “achieve the bigger vision: having one-to-one conversations across all of omnichannel touch points.” Every digital signal or interaction can be tagged, making it possible to respond to individuals, including the “influencers” that can make or break offerings.

One big issue is data ownership. As Ensighten’s Manion points out, data traditionally has been stored in vendor-owned cookies, which has made it more difficult to consolidate data across silos and define digital interactions with customers. Brands should own their data, he maintains: “Customer profiles are an asset that accrues to the benefit of the brands and not third parties.”

There is no question that the volume, velocity, and variety of data is only going to grow. Marketing teams need a vision for how data will be used to support relationships with consumers – the big picture trends and the smaller, more personal information about customers that can lead to custom experiences. Fortunately, the new paradigm powered by tag management and multichannel analytics tools will give marketers access not just to more data, but rather help them hone in on the right data at the right time.

Related reading