Virtually every Fortune 500 company has amassed huge volumes of data that could be used to help inform and improve their marketing activities. These data assets are ballooning rapidly and traditional data management - based on relational database architectures designed to quickly find one individual customer's data at time - just aren't up to the task. The ability to simultaneously bring together and make sense of all of the billions and trillions of data points is the process for a new approach to information called Big Data.
First, let's take a step back. A recently published McKinsey report defines Big Data as "datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze . . .." Today, Big Data is measured in petabytes - a number followed by 15 zeros. Imagine 100 football fields packed with four-drawer filing cabinets worth filled with documents and you get a sense of the incredible scale of a petabyte. But more important than this striking visual, Big Data is about the ability to put incredible volumes of unfiltered, unsampled data to work in real-time for a wide variety of ad hoc analytical purposes.
The new ability provided by Big Data radically redefines the way we store, manage, and use information. Thanks to the declining cost of storage and advances in distributed computing, a growing number of companies are able to rethink their relationships with information. Companies no longer need to think about building robust traditional database systems, creating multiple data backup copies, de-normalizing data, or making data schema changes. The move away from traditional databases and storage to the new, more cost effective, Big Data model (often referred as NoSQL), has accelerated over the past two years. Big Data has introduced the potential to fully unleash the power of advanced analytics and optimization by giving algorithms unfettered access to the broadest possible datasets.
This is where Big Data gets really exciting for digital advertisers. Suddenly, advertisers are able to create new possibilities, opportunities and value - all driven by Big Data - that were unimaginable even a few quarters ago. Let's explore some examples of these new abilities:
1. Providing transparency and trust of media consumption.
With every event-level data point captured from log files, media buyers (advertisers and ad agencies) can examine and analyze every detail of their customers' media consumption. The result is full transparency into their campaign performance. Because data can now be stored in a distributed and de-normalized state, marketers (and the technology that supports them) no longer need to trade data volume for query speed. The result is that queries and reports can be delivered at much lower costs - and at much higher velocities - than from a traditional relational database system.
2. Enabling audience targeting experimentation across media channels.
Having 100 percent of audience data always available in a "ready" state enables audience discovery, planning, and experimentation at scale across multiple media buying channels. Sophisticated optimization algorithms can be applied across a Big Data cluster to find, build, and test audience segments. Dozens of business rules and targeting conditions can be used without impacting system performance. Big Data also means unprecedented flexibility in dealing with various types of linking keys required for different media channels, such as display, video, mobile, search, email, and IPTV. Demanding this type of structural flexibility or rapid changes would have driven traditional relational database management systems to their knees. Thanks to new technologies powering Big Data, these system-killing capabilities are the norm for modern audience data management platforms.
3. Driving real-time budget and bid price optimization.
The most data intensive tasks in digital advertising involve computing real-time budget optimization across all addressable media channels and optimizing the price paid on every available ad impression in real-time. This can involved billions of ad calls, impressions, and calculations per second. We're seeing the online advertising industry following the same efficiency curve seen when the financial, retail, and manufacturing industries adopted the Big Data strategy.
4. Obtaining advanced analytics insights that can lead to new products and services.
The availability of petabytes of data for real-time analytical access also means people can ask questions they'd never been able to in the past. The answers can lead to new insights about customer behavior, media performance, channel performance and how they all might impact an advertiser's current business. These insights, in turn, can generate ideas for new products and services. As time to market and anticipation of consumer needs become the most critical elements for winning, insights gained from Big Data can give big brands a big advantage.
Most advertisers are sitting on top of an untapped treasure trove of audience data - they just haven't had the tools necessary to take advantage of it. In 2011, Big Data has gone from theory to reality in making this data both available and actionable, and it's driving tremendous marketing value for the advertisers who embrace it. If your company is not thinking about using Big Data to support your next digital advertising initiative, you may not be the next dinosaur but you might be on your way to becoming the next mainframe.
Meet Your Favorite ClickZ Contributors
Many of ClickZ's leading expert contributors will be at ClickZ Live, the new online and digital marketing event kicking off in New York (March 31-April 3). Hear from the likes of: Jeremy Hull, Lisa Raehsler, Andrew Goodman, Bryan Eisenberg, Mathew Sweezey, Aaron Kahlow, Stephanie Miller, Simms Jenkins, Jeanne S. Jennings, Dave Hendricks and more!
As Chief Technology Officer at Turn, Xuhui Shao focuses on the power of optimization, machine learning, and advanced analytics solutions in driving new business models, products, and services across all industries. Xuhui is responsible for architecting the machine learning and optimization technology to deliver the most effective data-driven digital advertising in the world. He is passionate about the dynamic online advertising community and works closely with industry leaders developing data transparency and consumer privacy protection.
For the last 12 years, Xuhui has practiced research and development in machine learning, statistical theory, and computational intelligence for Fortune 100 companies in various industries from banking, finance, online retailing, healthcare, insurance, marketing, and online advertising. As the lead inventor and co-inventor of three awarded patents in the areas of advanced analytics and optimization, Xuhui is a recognized expert in harnessing data and transforming analytics into actionable insights and optimization strategies.
He earned his bachelor's and master's of science degrees from Tsinghua University, Beijing, and his Ph.D. in electrical engineering from the University of Minnesota.
March 19, 2014