Emotional AI platform reveals that smiles are down 32% due to COVID-19

Headquartered in London, Realeyes is an eye-tracking and emotion measurement platform that uses AI and machine learning to gain insight into human behavior and expression. Their clients include Buzzfeed, Coca-Cola, Conde Nast, eBay, Mars, and Publicis Groupe, among others.

Realeyes uses front-facing cameras, computer vision and machine learning technologies to detect attention and emotion among opt-in audiences as they watch video content.

ClickZ recently spoke with Max Kalehoff, VP of Marketing & Growth for Realeyes to discuss the company’s innovative technology and the capabilities they bring to marketers and publishers.

A background in digital measurement

Kalehoff learned about Realeyes after co-leading a panel presentation with Realeyes’s CEO at the Sustainable Brands Conference in 2017. He started working with the company this past November.

Kalehoff started his career at start-ups that focused on analytics and new intelligence in marketing, including Media Metrix which was acquired by comScore in the early aughts.

He was part of the founding management team at BuzzMetrics, which was acquired by Nielsen and became Nielsen Digital. Kalehoff was also on the founding management team at Clickable, a search technology company.

“I’m really passionate about the marketing space,” says Kalehoff, “I think it’s interesting, particularly the analytics which is the glue that makes it all work.”

Kalehoff recognized that a big piece of what was missing from analytics was the notion of measuring attention. He started a blog about it with the Advertising Research Foundation (ARF) back in 2006, one of the first marketing blogs that examined attention versus engagement.

“I think attention is a big, uncharted opportunity,” explains Kalehoff. “Realeyes brings a fresh perspective to attention measurement by applying emotion AI and computer vision to help understand attention, particularly with video and with interactive spaces.”

The AI technology behind measuring emotion

Realeyes uses the front facing cameras on devices such as laptops, mobile phones, and tablets. With the user’s permission, Realeyes enables a lightweight algorithm, which applies basic coordinates over the user’s face and the system uses these coordinates to measure facial cues.

Example of the Realeyes’s system mapping Kalehoff’s face

The Realeyes system can detect major emotions including happiness, confusion, contempt, disgust, empathy, fear, and surprise.

The technology is based on a branch of psychology pioneered by Paul Ekman, in the 1970s and 80s. Ekman created facial coding, which is a technique that psychologists use to detect emotion using passive observation of somebody’s face.

Says Kalehoff, “It’s an established school of psychology. We use trained psychologists and human annotators to review media like videos or memes and other experiences that happen digitally. We have these human users tag what emotions are happening at certain points. Over time the machines learn and become very precise at understanding the emotion.”

About a year ago, Realeyes pioneered attention monitoring as a fundamental breakthrough in their technology. Attention monitoring measures a user’s engagement with the content they’re watching and applies different levels of intensity based on their interaction with the stimuli that they are receiving.

Attention can be fortified or degraded based on the level user distraction — are they looking away, drinking something, chewing gum, etc.?

“In marketing and advertising, the fundamental element is attention,” explains Kalehoff. “It’s what people buy and sell. If you’re a publisher, you’re selling attention. If you’re a marketer or a media buyer, you’re buying attention.”

Realeyes uses the same panel responder platforms that Nielsen, Kantar and Ipsos use. Marketers can  upload a video to the Realeyes website which is then viewed by panel responders who have integrated Realeyes’s software to their device.

“We deliver a forced exposure video and the panelists go through a consent process so they understand that we’re going to turn their camera on, the purpose of what we’re doing, and that we’re looking to understand their response to ads and other media.”

Applying emotional AI to marketing

Today, Realeyes is using their emotional AI technology to help marketers with attention measurement. Understanding attention in an ad creative enables marketers to predict which content or which creative will earn them the most attention once they put it live in market.

The emotion classifiers enable the system to make predictions about the level of attention a creative will receive before they ever go into the market or get pushed by paid media.

“This is a very direct benefit of the advancements in the accuracy of emotion detection,” says Kalehoff. “Another important benefit is that we’ve advanced our training set using new classifiers to accurately detect the emotion of Asian people who represent a large portion of our world. Having the ability to be relevant and apply the technology in other countries, including Asian countries, is a huge part of the advancement in the emotion classifiers.”

Going forward, Realeyes is taking their emotion AI to the live experience. They have beta products with various major video platform and gaming companies where we’re they’re using attention detection on live in-app streaming, gaming, and mobile experiences.

Says Kalehoff, “Think of your favorite video platform being able to understand the attention of users through actual facial coding or imagine if you’re a gaming streaming company that’s able to understand if users are paying attention to the streaming of video game celebrity Ninja, or are they focused on the game  they have going on in another window?”

Realeyes’ classifiers enable them to use attention to predict outcomes from advertising creative. The emotion classifiers enable users to branch out into new markets and cultures. The future of this technology is in measuring live experience.

Measuring success in smiles

Realeyes recently published a case study which focused on the monitoring of norms. Kalehoff explains.

“As COVID-19 took hold, we did an analysis on the ‘share of smiles’ or happiness that consumers have as they watch video,” says Kalehoff. “We found a 32% decrease in smiles as a result of the COVID-19 experience. The reduction in smiles was driven by certain industries, like travel, hospitality, consumer electronics and luxury goods. This is significant because it gives context that consumer sentiment is changing and possibly at a grand scale.”

Realeyes also did a case study with mobile digital ad company Teads which delivers interactive mobile ad experiences across premium publishers on a global scale.

Teads  worked with Realeyes to use emotion classification and attention measurement to build out their attention prediction model. This model enables the system to look at existing creative or content, test that creative on a forced exposure test, and predict the attention in viewing that those ads will accrue in the future.

“That’s very powerful intelligence for both the marketer and a publisher to have,” says Kalehoff. “Teads is a tech publisher and they’re using this intelligence directly to drive the performance and better outcomes of their advertisers.”