Emerging TechnologyAR & VRYou know what AR is. But do you understand how it works?

You know what AR is. But do you understand how it works?

AR is one of today's hottest technologies, allowing consumers to virtually try on makeup and turn themselves into dogs on Snapchat. Here's how it works.

Augmented reality is one of the hottest technologies of this year and last year, and sure, 2017, too. Of course, it is, AR is a staple of modern mobile apps. It allows consumers to virtually try on clothes and makeup, visualize furniture in their homes, catch Pokémon, turn themselves into dogs on Snapchat, and even look at Facebook ads.

We write about AR all the time, but our focus is on how it’s used. Now, we’re going to delve into how it actually works.

Intro to computer vision

In the most basic definition, AR involves superimposing sensory enhancements such as images or audio over what you actually see in the physical world. You probably knew that already, but you may not understand computer vision, where this begins.

Computer vision is an interdisciplinary scientific field that refers to computers’ understanding of digital videos and images. In other words, digitizing the human visual system. Computer vision acquires, processes, analyzes and understands these images, the way your retina would, and extracts their data, turning them into numbers and symbols. This is how Facebook recognizes people when suggesting photo tags, how you’re able to deposit a check with the Bank of America app, and how you can turn yourself into a taco on Snapchat.

AR - Snapchat filters

Let’s use the Snapchat filter as an example. When you use your camera, every frame is like a tiny image. Computer vision processes those images — in this case, your face. There are a few universal truths to the human face. Our eye regions are always darker than our upper cheeks, while the sockets are always darker than our foreheads.

With that information, computers scan image data and detect contrasts between lighter and darker pixels. Those pixels lead to facial recognition, but only if they’re facing forward. We’ll get back there.

What computer vision means for AR

Computer vision comes into play with AR because, in order for the technology to work, the computer must also understand the context of the physical world. Say you scan a Pizza Hut box to unlock AR content. Computer vision processes the position of the box, any images printed on the box and which direction it’s facing, and what else is in the frame.

Pizza Hut’s unbox AR campaign

Of course, the pizza box will have been embedded with a predefined marker. When the AR system recognizes that box, it captures the image with your smartphone camera. From there, it scans the image with sensors to identify exactly where to overlay the additional information on the box.

Next, the AR solution overlays the content on that spot. This must align precisely over the original frame, which explains why Snapchat filters disappear when you move your head. Together, these form a complete image, a combination of the real world and AR data.

There is also position-based AR, in which the app also utilizes GPS data and the mobile device’s compass, accelerometer, and gyroscope. The latter two determine your direction and orientation, respectively. Putting them in smartphone terms, the accelerometer is how your smartphone knows which direction you’re facing when you look at a map, while the gyroscope allows you to change the device’s orientation by rotating it.

AR - Chicago bus stop

Position-based AR uses geolocation, rather than the embedded marker, to display the content. Instead of pointing your phone at the Pizza Hut box, you can point it at a sign outside the restaurant, which triggers additional information, such as hours of operation or the menu. Last year, Perficient Digital Labs did something like this with Chicago bus stops.

Wrapping up

AR is live so this occurs every time the camera detects a new frame. This should theoretically occur in 16 milliseconds, as smartphones typically work at 60 frames per second. The AR frame is generally delayed by far more than that but it’s also far faster than the human brain can compute.

Either way, it’s helpful to understand because AR is far more popular than its cousin, virtual reality ever was. In March, eMarketer forecasted that 42.9 million Americans — 13% of the population — will use the technology at least once a month in 2019. Meanwhile, AR is expected to have a whopping 60% more users.

AR is also more practical. It doesn’t require a whole new device that blocks out the rest of the world; it’s already part of the device you already use to block out the rest of the world.

Subscribe to get your daily business insights


US Mobile Streaming Behavior
Whitepaper | Mobile

US Mobile Streaming Behavior


US Mobile Streaming Behavior

Streaming has become a staple of US media-viewing habits. Streaming video, however, still comes with a variety of pesky frustrations that viewers are ...

View resource
Winning the Data Game: Digital Analytics Tactics for Media Groups
Whitepaper | Analyzing Customer Data

Winning the Data Game: Digital Analytics Tactics for Media Groups


Winning the Data Game: Digital Analytics Tactics f...

Data is the lifeblood of so many companies today. You need more of it, all of which at higher quality, and all the meanwhile being compliant with data...

View resource
Learning to win the talent war: how digital marketing can develop its people
Whitepaper | Digital Marketing

Learning to win the talent war: how digital marketing can develop its peopl...


Learning to win the talent war: how digital market...

This report documents the findings of a Fireside chat held by ClickZ in the first quarter of 2022. It provides expert insight on how companies can ret...

View resource
Engagement To Empowerment - Winning in Today's Experience Economy
Report | Digital Transformation

Engagement To Empowerment - Winning in Today's Experience Economy


Engagement To Empowerment - Winning in Today's Exp...

Customers decide fast, influenced by only 2.5 touchpoints – globally! Make sure your brand shines in those critical moments. Read More...

View resource