Advertising's Toughest Tech Problems, Part 1

There are many technical challenges in the interactive advertising space. Some could be resolved by the industry changing behavior, some by publishers working cooperatively. There are implications in how our market evolves and how advertisers will adopt interactive advertising if we don’t resolve the challenges we face.

Inventory Prediction

One of the biggest, hairiest tech problems is inventory prediction by publishers’ ad servers. Every publisher that sells guaranteed impressions must be able to have its sales team look up available future impressions and book them for sale. Yet the technical complexity of the algorithms that predict future available inventory is intense.

Even predicting the number of visitors across site pages (with knowledge of the ad units on each page applied) is a huge technical challenge. Variables such as seasonality can throw a prediction off, never mind national events such as disasters, storms, wars, and scandals. Additionally, most publishers sell inventory with various kinds of targeting and frequency capping, as well as use multiple sales methods, such as CPC (define) and CPM (define).

Imagine trying to predict how many impressions will appear on MSN and Yahoo on December 19, 2007. Let’s say an advertiser wants to buy 100,000 impressions on their shopping sites that would run that day. If the only variable is impressions, there’s a decent chance both portals will be able to deliver the ads they sold.

But the advertiser wants to buy targeted impressions. What’s the likelihood both portals will be able to deliver these impressions? Not as high. Predicting not only how many people are coming to the site that day but also what geography they’re coming from, or what demographic segment they’ll represent, or what behaviors they’ve exhibited in the past 60 days, is very hard. Throw in even more variables, and it becomes nearly impossible to solve.

This is a big engineering problem, one that will likely win mathematics awards when it’s solved. There aren’t many companies with the technical resources to crack it. And it afflicts the entire industry, as we’re frequently seen as flaky and too complex by traditional buyers.

It shows a hard technology limitation to how you should expect the industry to evolve. Inventory prediction is too hard a problem for small companies to resolve. It’s a mature technology problem, not one that typically gets solved by a startup. Of course, those could be famous last words. But the real nugget here is that startups should find other problems to solve and hook into publishing systems that are likely to resolve this issue when they need inventory prediction.

Targeting

People talk about how targeting technologies will solve all the industry’s problems. The idea is that since people dislike advertising primarily because it isn’t relevant to them, we just need to learn a lot about the audiences on our sites and deliver targeted, relevant ads to them. Then, everyone will love advertising. We can serve ads with impunity.

Targeting causes serious inventory prediction problems, of course, one of the reasons I chose to write about both topics in this column. But it’s so valuable that we really need to go down this path.

Let’s look at behavioral targeting first. You use technology to create an anonymous identifier for each visitor, removing all personally identifiable information about that person. Then you track this anonymous person as she moves around various Web pages. Based on the pages’ content, you form a profile of this anonymous person to put advertising in front of them that’s more valuable.

Once you have enough information about enough unique profiles, you can place each profile into a behavioral segment that can then be sold. You might imagine profiles are segmented out in ways that help sell advertising, such as automotive buyers.

The main problem with behavioral targeting is for it be valuable, you need to know an immense amount about a large audience. Yes, there are many ways companies ensure visitors’ privacy, so there are no privacy implications for any of the significant companies doing this work today. But if we’re to get real value from behavioral targeting, we must know a lot about a lot of anonymous people. What’s happening on one site isn’t enough value; we need cross-Internet behavior.

In the past, the only way people could get at this broad audience was with client-side technology such as Claria (Gator) and WhenU. But consumer and industry backlash against these installed clients (and particularly their preferred ad vehicles — pop-up ads) eventually drove them to new business models. Today, we see companies like Tacoda building vast knowledge bases of anonymous audience profiles across huge swaths of the Web. The ability to leverage large, broadly informed behavioral segments will really help.

Other types of targeting vary, but include the other two broadly available types: geographic and demographic.

Geographic targeting is typically done with reverse-IP lookup. Basically, there are databases of all IP addresses mapped against the geographic locations of all the computers hooked to the Net. This is relatively accurate where the location is known, but some IP addresses are still cloaked geographically.

Demographic targeting is a bit trickier. Some ways we build out targeting involve applying known data sets against each other. For instance, if we have geographic targeting data we can apply generally known demographic data against the known location. This is a bit funky, as you don’t really know for sure each impression is delivered to a person matching the description you have. But it does drive better results statistically. And of course, some publishers have demographic data from user registration data.

The big problem the industry faces with targeting isn’t actually implementation. It’s the fact we don’t have enough advertisers placing ads in the market to make the majority of ads are relevant on an individual basis. Typically, only about 20 percent of any audience meets targeting parameters that match advertising that’s been purchased in any given system.

Until we can aggregate advertisers across multiple sites with lots of volume and find ways to price and sell targeted audiences in ways that advertisers value, we’ll have a problem getting targeting adopted to the point at which relevancy is affected.

In part two: more deep technology problems, including frequency capping.