Emerging TechnologyImmersive AR is coming to a movie theater near you

Immersive AR is coming to a movie theater near you

Computer vision platform, Resonai, is bringing augmented reality to physical spaces, including movie theaters, shopping malls and retail.

30-second summary:

  • Resonai enables building and business owners to turn their physical location into an intelligent augmented reality (AR) environment.
  • Resonai recently announced a partnership with CJ 4DPLEX, a leading cinema technology company, to create a mobile RPG which activates an AR experience whenever a user enters a participating movie theater.
  • Resonai is an AI startup that brings visual perception to machines, giving them the ability to understand the environment.
  • Vera is Resonai’s product for the commercial real estate industry. It’s an AI & AR platform which converts  physical spaces into intelligent 3D environments. 
  • Resonai scans a building once and creates a highly intelligent digital twin available in the AR cloud that communicates with Vera mobile applications. 
  • Applications, in addition to those developed by Vera, can be developed by third-party creators. Once Vera creates an intelligent 3D mesh in the cloud, users can run a variety of AR applications and even remotely inject new AR content. The Vera engine matches these applications to a specific interior’s properties, functionality and physical properties.
  • Any mobile phone that runs on top of AR Core or AR Kit will function in a Vera-enabled environment. AR Core and AR Kit comes with most phones that were manufactured in the last few years.

Resonai, a computer vision company, launched their Vera™ platform in November 2019 which allows building and business owners to turn their physical location into an intelligent augmented reality (AR) environment.

Vera is an AR and artificial intelligence (AI) technology that can convert any physical space into an intelligent digital environment.

It scans commercial real estate properties to create a digital twin which enables operators to drop customized augmented reality experiences into the physical location for consumers to access on their mobile devices.

Example of a digital clone of a building—Image Source: Resonai

Resonai recently announced a partnership with CJ 4DPLEX, a leading cinema technology company, to create a mobile RPG which activates an AR experience whenever a user enters a participating movie theater.

The game will initially be available in Korea and was demoed for the first time at CES 2020.

ClickZ spoke with Resonai’s CEO, Emil Alon, to learn more about how the Vera platform works and its implications for companies like CJ 4DPLEX, which is in the entertainment space.

A brief history of Resonai and its founder

Resonai is an AI startup that brings visual perception to machines, giving them the ability to understand the environment.

Their patented 3D visual engine understands the precise geometry and structure of 3D content and can search vast amounts  of 3D data in real time, coupled with precise positional tracking/ localization without the use of markers makes up the core of their technology.

Emil Alon founded Resonai in 2013. The company’s core technology team is based in Israel with the business development and marketing teams are based in Palo Alto.

Resonai is Alon’s third startup. His background in math, science, and physics led to him start companies in the field of AI, building personalized search engines that deploy AI to learn about user intent and serve much more relevant information.

His first startup was Collarity, a search technology company based in Palo Alto.

He also founded Pebbles Interfaces, a VR/AR sensing platform that utilizes 3D sensing technology for human gesture recognition. Pebbles Interfaces was acquired by the Oculus Team at Facebook.

“Resonai is a combination of my previous experience within search relevancy and understanding semantics at Collarity and the importance of 3D depth information to enabled devices,” explains Alon. “The combination of both technologies naturally led me to establish Resonai, which was founded to solve major complex problems in computer vision and to enable devices such as robots to understand the world as we humans do—and, in many cases, better.”

Understanding the Vera™ Platform

Vera is Resonai’s product for the commercial real estate industry. It’s like an operating system for physical spaces.

We are entering an age where the digital and physical are beginning to blend, not only because of humans, but because of sensors, devices, autonomous cars, robots and any type of intelligent agents that are operating within our physical spaces.

Says Alon, “Vera is an operating platform to synchronize all those agents in real-time with respect to a facility’s actual physical space—including devices, functionalities, and everything that exists within the space. This is what the operating system’s infrastructure is all about.”

The Vera platform is the foundation for applications that can be written using Resonai’s SDK and is being utilized  for various use cases such as; in building operations for more efficient maintenance, in retail to create engaging shopper experiences and in store shopper navigation and in entertainment for gaming or to extend brands into physical locations.  

Says Alon, “The separation between Vera the platform and third party applications is very important. It’s similar to how a mobile phone operating system works in correlation with the app store.”

Vera hosts applications via two important steps:

  1. Resonai uses a high quality scan to create an intelligent 3D mesh of the given structure. The digital mesh is stored in the AR cloud. From there, any device can access the Vera applications. In addition, building administrators are able to remotely inject or update AR content into the physical space.  
  2. Using Vera’s SDK, applications can be  developed by third-party creators. Any device or user operating in the Vera enabled environment can then communicate and operate these applications. 

“Developers don’t need to know the exact geometry (windows, corridors) of a specific building,” says Alon. “They can build the applications based on Vera’s SDK using higher level semantics that are not geometrically specific to the space. Our algorithms match the application to a specific interior. The owner of the interior simply needs to index the point of interest that they want the application to address. For example, a retail store can provide a shopping list capability for shoppers and the indoor navigation will direct the shopper to the exact position in the store that’s relevant to where the user can find the product and also provide relevant promotions and recommendations in AR.”

Onboarding with Vera works two ways. It can be web-based through a QR code or push notification, so when a user enters a vera-enabled location, they receive a push notification indicating it’s Vera-enabled.

The user is then able to access all Vera applications relevant to that space. Examples of applications include navigation, access control of buildings, entertainment and brand promotion, IoT controllers to manage smart devices in a building and many others.

Another option is that if a user has a Vera-enabled application installed and enters a Vera-enabled building, the application will launch on the user’s device  and the user will immediately see the AR experience and access its functionality.

Industry-specific usage for Resonai’s AR technology

There are currently four main applications for Resonai’s technology, as follows:

  1. Retail: In the retail sector, there is demand for navigation, digital concierge, the ability to search for a specific product, and the ability to personalize the shopping experience. “Vera-enabled businesses have access to analytics that give them an understanding of where shoppers are going in the store,” explains Alon. “What are the hot spots, what are they looking for, and how are they interacting with products on shelves? They can also add what we call “digital assets”—new layers of real estate with advertisements, personalized content, etc.”
  2. Maintenance and operations: Applications include an AR-based ticketing system that makes it easy to report a problem using your phone. Technicians get automatic navigation to the specific place that needs service, can automatically retrieve maintenance historical data and can see hidden layers of a building with the use of their mobile phone.
  3. Tutorials and onboarding of new employees: Many companies have machinery or complex devices and want to overlay instructions or tutorials on top of those devices. Employees can use their phone or tablet to see the actual information and processes embedded on top of the devices for use as guide or walkthrough on how to operate, maintain or fix a device.
  4. Entertainment: With digital twin technology, the Vera engine can understand exactly what the user is looking at down to the level of pixels. This is unprecedented in terms accuracy and robustness. Based on this information, they can create an immersive experience in the physical space. This is the basis for Resonai’s partnership with CJ 4DPLEX. “CJ 4DPLEX is a global company that operates within hundreds of high-end theaters around the world,” explains Alon. “They want to expand into AR gaming by building a platform to host AR games within the physical spaces they’re operating. Resonai is entering into a long-term partnership with them and will supply the platform that gives the modeling, semantic realization, tracking system, and SDK to develop the application, games, and content that will be immersed and embedded in these spaces.”

Any mobile phone that runs on top of AR Core or AR Kit will function in a Vera-enabled environment. AR Core and AR Kit comes with most phones that were manufactured in the last few years.

The marketing implications for AR

Resonai’s partnership with CJ 4DPLEX is the first concrete example of how a AR presents a brand new channel for marketers.

The cinema technology company is developing a 4DX AR platform which creates a realistic AR experience, incorporating elements of the mobile RPG game inserted into the physical space via the Vera platform.

The AR version of the game can be played in the physical theater space, without any lag.

With this technology, the possibilities are exciting for marketers, particularly in the retail sector.

Companies can create promotions that use AR which is visible in a physical location.

The tech can be used to gain a new level of insight into shoppers’ store behaviors, traffic patterns, and their interaction with merchandise.

Examples of AR-driven applications include:

  • Digital coupons or promotions when a person enters a store or business venue
  • Pop–up ads or videos in a physical stores or venues
  • Marketing materials which encourage people to visit other locations within a store or venue

Marketers will gain valuable data about how users navigate within a Vera-enabled building including where they’re spending the most time.

From a retail perspective, they’ll be able to see what merchandise, store departments, and promotions that shoppers interacted with.

Alon predicts that commercial real estate is going to change dramatically in the years to come with AI and AR becoming a major part of real estate as we know it.

“We see Vera as a platform that turns digital creative power into actual reality, enabling people to experience applications and content in a synchronized manner. As a trend, AR will help us consume digital information in a much more natural way within the indoor environment.”

Subscribe to get your daily business insights

Whitepapers

US Mobile Streaming Behavior
Whitepaper | Mobile

US Mobile Streaming Behavior

5y

US Mobile Streaming Behavior

Streaming has become a staple of US media-viewing habits. Streaming video, however, still comes with a variety of pesky frustrations that viewers are ...

View resource
Winning the Data Game: Digital Analytics Tactics for Media Groups
Whitepaper | Analyzing Customer Data

Winning the Data Game: Digital Analytics Tactics for Media Groups

5y

Winning the Data Game: Digital Analytics Tactics f...

Data is the lifeblood of so many companies today. You need more of it, all of which at higher quality, and all the meanwhile being compliant with data...

View resource
Learning to win the talent war: how digital marketing can develop its people
Whitepaper | Digital Marketing

Learning to win the talent war: how digital marketing can develop its peopl...

2y

Learning to win the talent war: how digital market...

This report documents the findings of a Fireside chat held by ClickZ in the first quarter of 2022. It provides expert insight on how companies can ret...

View resource
Engagement To Empowerment - Winning in Today's Experience Economy
Report | Digital Transformation

Engagement To Empowerment - Winning in Today's Experience Economy

1m

Engagement To Empowerment - Winning in Today's Exp...

Customers decide fast, influenced by only 2.5 touchpoints – globally! Make sure your brand shines in those critical moments. Read More...

View resource