TechTalk: Augmented, virtual & mixed realities

The risk chain grows more complex

June 15, 2017
| United States

By Jose Manuel Mercado

Real Worlds: When Pikachu Fight Back

It took a little yellow digital monster to make augmented reality (AR) and virtual reality (VR) technology a mass-market ‘product’ in 2016. The global impact of Pokémon Go may lead to an uptick in investment in similar mobile game-based products this year. However, AR/ VR technology has more to offer the world than just finding fantastic beasts, and the mixed reality (MR) applications that will emerge offer as many risk-based challenges as financial and efficiency rewards.

Virtual Reality, Augmented Reality and Mixed Reality: what’s the difference?

While the concept of VR has existed since the 1960s, the popular perception of the technology has been colored by speculative or inaccurate representations in the media of its potential as a useful application. In many ways, this perception will continue because the user interface to VR applications continues to take the form of cumbersome headwear and limb-based prosthetics, and the high cost of hardware and software will shut many potential users out of the market. By contrast, AR is a lower-cost variation and, as the success of Pokémon Go demonstrates, can offer a more social or group-shared experience that has more appeal.

Virtual Reality Augmented Reality

A digital environment in which the user’s basic senses (usually sight and hearing) are cut off from the real world. Expensive headsets, controllers, seats and, base stations/consoles are needed to fully exploit VR.

Immersive applications include remotely attending a live (or recorded) concert, participating in a simulated sporting event, socializing through avatars, emotional therapy, etc.

Digital content is placed on top of the physical world when viewed through a low-cost device such as a smartphone, tablet or headset. Additional location- or context-based information can also be unlocked, if available.

Applications include passive and interactive gaming, physiotherapy, watching video guidebooks at a tourist location, buying goods or services in store, and paying automatically via contactless or location-aware technology.

Source: BMI Research

VR/AR concepts and techniques are already in widespread use in multiple industries, ranging from hospitality and media services (art exhibitions, press launches, etc.) to the sales sector (virtual sales brochures ‘attached’ to products in shops), construction and infrastructure (digital models of planned offices or industrial sites can be toured before committing plans to paper) and beyond, into the automotive, health care and social services sectors. The cost-savings benefits of VR/AR are clear, lowering risks to investment, and increasing the prospects of anticipated returns. But there are risks, too, and these become more nuanced and more serious as applications are transposed into AR and, increasingly, MR environments.

Mixed Reality: The New Paradigm

This year, we’ll see developers move to a new paradigm, mixed reality (MR). This hybrid concept leverages the core elements of both VR and AR to provide more complex and nuanced applications that will have a deeper and longer lasting impact on industries. With highspec sensors and advanced computing power, MR systems will scan a user's environment and recreate it as a 3D map to show realistic digital content. Values such as weight and shape can be assigned and the data transmitted back to the user’s headset, body armor, and gloves to create a more realistic and immersive real-time interactive experience (haptic interfacing).

Games and entertainment are being used to test the viability of this technology. Platforms such as Microsoft's Hololens have been the highest profile use cases for MR in the last 12 months, with Microsoft working with the National Football League (NFL) to help sponsors, teams and advertisers better connect with viewers through valueadded content and context-based information.

At a cost of about $3,000, Hololens remains an enterprise- and engineering-focused product. To make this a more mass-market proposition will involve making user interface hardware smaller and more affordable, and leveraging hyperscale data processing solutions such as cognitive or fog computing to avoid issues of data transfer latency in solutions, depending on real-time interactivity.

Spearheaded by the Sony Playstation VR (which sold more than 900,000 headsets between October 2016 and February 2017), the HTC Vive and the Facebook-owned Oculus Rift, as well as freemium AR applications and high-spec immersive machines, the market for digital reality hardware is expected to be worth US$110 million in 2020, up from US$10 million in 2016, according to IDC, with VR leading the way.

The main risks associated with VR, AR, and MR are many and varied and become more complex with each iteration.

Increasingly complex risk chains

The main risks associated with VR, AR, and MR are many and varied and become more complex with each iteration.

End users might become over-reliant on the accuracy of data derived from virtual simulations and find themselves in a situation, such as computer-assisted surgery or house construction, where a less-than-ideal positioning of a scalpel or a water pipe may be necessary to accommodate an unexpected obstacle.

This is minor compared to the risk posed by data security. The high cost of VR solutions means that many bespoke systems are being created by end users to keep costs down for their particular projects. Obtaining core software from open source platforms runs the risk of inadvertent exposure of coding to third parties or allowing sensitive information to be shared by accident or malicious design.

Among others, the U.S. Army has an open source software called OpenSimulator. This package offers features that enable virtual world 'owners' to allow users to travel between different worlds or to upload their own content and have complex interactions. At best, a simple password is employed to protect access to servers powering these worlds, which is a fairly insecure process. The Army has had to build additional security to protect information such as troop deployment scenarios and logistics frameworks.

In more complex platforms, users' avatars are used for identification. Should those avatars be copied, stolen or adjusted, sensitive information could be removed, deleted or manipulated, with devastating results. However, adding further layers of security bears additional costs and may interfere with the smooth running of VR systems.

Mind where you’re pointing that thing!

From a mass-market perspective, there’s potential for physical injury (e.g., headsets can lead to falls due to disorientation caused by lags in movement in the real world and the virtual world), addiction to use, and accidents due to impaired judgement of speed, distance and response times when driving, for example.

AR is likely to see greater use outside the home, where potential dangers are even greater. These were best illustrated after the release of Pokemon Go in 2016, when players attempted to reach Pokemon in difficult-to-reach locations, injuring themselves in the process, or being assaulted and robbed when going to a Pokestop reported to spawn rare creatures. Privacy infringements have also increased as a result of Pokestops or Pokegyms being situated in private properties without owners’ knowledge or consent. User agreements are insufficient to cover all possible permutations of misuse, and it’s far from clear where liability ultimately lies.

Most significant, once the cost and micro-scaling of MR hardware comes down to a point where mass-market roll out becomes possible, will be the extent to which users will knowingly share personal data with MR systems. The potential for MR to become as ubiquitous as the smartphone means that MR devices could track huge amounts of personal data every day. Passive interactions with, for example, items on a shelf in a store as wearers of MR glasses scan for the most attractive products would be of immense value to advertisers, retailers, producers and their rivals. Integrating contactless payments into these devices and the potential for voice-based artificial intelligence (AI) systems to also be integrated into a wearable device raises those risks still further.

Laws and regulations will lag behind technology

Laws and regulations will continue to lag behind technological development and adoption, and it’s likely that current individual, corporate and government insurance toolkits will be ineffective in dealing with new forms of crime and misuse. This will affect individuals just as much as the providers of the hardware and software that will power these digital empires.