The Magic of Machine Learning in Real Time

  |  August 3, 2010   |  Comments

In digital advertising, the machine learning prediction boils down to two parts: identifying your target audience and reaching them efficiently.

Part of the magic of real-time bidding is found within machine learning. This involves using sophisticated algorithms to "learn" complex patterns based on large amounts of data in order to make optimal advertising decisions. The importance of machine learning is cost avoidance and value creation. Cost avoidance is simple to understand: data-driven optimization strategies help reduce waste by identifying the most relevant impressions while selecting the best ads (better creative message, better offer, etc.), which in turn improves performance and ROI (define). Value creation, on the other hand, happens when buyers and sellers of commoditized offerings are more efficiently brought together for a transaction.

To create machine learning magic, two ingredients are required: scale and prediction. Scale speaks to the need to make more users/impressions available through the auction marketplaces, and increase the number of advertisers bidding on these users. The bigger the scale, the more sophisticated the data-driven prediction can be. The second ingredient refers to the idea that prediction needs to be "accurate enough." Amazon and Netflix have demonstrated that when you provide an accurate prediction of what consumers want, the business grows in two ways:

  1. Better inventory control and more purchases/utilization.
  2. Better targeting becomes a custom delight feature when it's perceived to be quite accurate by average consumers.

The ability to deliver relevant choices in real time based on what consumers reveal about themselves creates a virtuous cycle:

Ads/recommendations become more accurate → consumers are willing to share additional information about themselves → the machine learning algorithm for ads/recommendations becomes even smarter

In digital advertising, the machine learning prediction ultimately boils down to two parts: identifying your target audience and reaching them efficiently. The first part requires machine learning at the user level to learn the most optimal audience segments to target; the second requires machine learning to drive real-time bidding strategy with precision. For instance, demand side platform technology allows advertisers to have global control over how many times each user sees the ads (i.e., frequency capping) and how they see them. This begs the obvious question, "what is the optimal number of ad repetitions?" The answer might be an average of five times over a period of seven days. Problem solved? Not quite.

To begin to solve this for marketers, machine learning analyzes the data of users who have and have not responded to the ads after each impression. The next step is to dig deeper to discover how attributes of the ad impressions and that of the users affect the optimal value. Impressions having higher influence increase the effective frequency count; while certain types of users requiring a higher numbers of ad repetitions reduce the effective frequency count. The outcome of each ad impression is also probabilistic, or in other words, gives the right answer with high probability, but not certainty. This requires the model to dynamically evaluate the expected value of each impression in real time to adjust the bid on a continuous scale.

From an auction market standpoint, value creation also depends on the number of smart buyers. A room full of bidders with over-simplistic bidding models will quickly realize that real-time bidding isn't working for them. Bidding inefficiently not only affects the availability of desirable inventory for other bidders, but also reduces overall ad relevance (a negative user experience) at large.

There are a number of general steps you can take to leverage machine learning to encourage value creation. In the order of complexity:

  1. Collect every bit of data on the advertiser side and protect them. For example, connect Web analytics data and online/offline CRM (define) data and apply them to the optimization of online advertising.

  2. Analyze the data; build machine learning models to leverage the power of these data points; predict who your best customers are; predict the best offer and creatives for different customers based on what you know about them.

  3. Invest in explanatory model (such as attribution) as much as predictive model.

  4. Integrate creative optimization with audience optimization.

  5. Model cross-media (search, display, mobile, etc.) interaction and optimization.

If executed correctly, marketers will reap the benefits of real-time bidding and data-driven advertising. Better ROI and value in the marketplace is here, but it's up to marketers to learn from the data, make predictions, and leverage those insights for optimization. Put your machine to work and let the magic happen.


Xuhui Shao

As Chief Technology Officer at Turn, Xuhui Shao focuses on the power of optimization, machine learning, and advanced analytics solutions in driving new business models, products, and services across all industries. Xuhui is responsible for architecting the machine learning and optimization technology to deliver the most effective data-driven digital advertising in the world. He is passionate about the dynamic online advertising community and works closely with industry leaders developing data transparency and consumer privacy protection.

For the last 12 years, Xuhui has practiced research and development in machine learning, statistical theory, and computational intelligence for Fortune 100 companies in various industries from banking, finance, online retailing, healthcare, insurance, marketing, and online advertising. As the lead inventor and co-inventor of three awarded patents in the areas of advanced analytics and optimization, Xuhui is a recognized expert in harnessing data and transforming analytics into actionable insights and optimization strategies.

He earned his bachelor's and master's of science degrees from Tsinghua University, Beijing, and his Ph.D. in electrical engineering from the University of Minnesota.

COMMENTSCommenting policy

comments powered by Disqus

Get ClickZ Media newsletters delivered right to your inbox. Subscribe today!



Featured White Papers

2015 Holiday Email Guide

2015 Holiday Email Guide
The holidays are just around the corner. Download this whitepaper to find out how to create successful holiday email campaigns that drive engagement and revenue.

Three Ways to Make Your Big Data More Valuable

Three Ways to Make Your Big Data More Valuable
Big data holds a lot of promise for marketers, but are marketers ready to make the most of it to drive better business decisions and improve ROI? This study looks at the hidden challenges modern marketers face when trying to put big data to use.