Virtual reality (VR) and augmented reality (AR) have proved that they have the potential to change the way businesses use technology. Imagine the impact of being able to combine the two – being able to effectively merge the real and digital worlds.
Enter mixed reality (MR), technology that combines the virtual environment with the real world. Experienced via a head-mounted wearable display, users gain a real-time view of actual surroundings combined with an overlay of intelligent virtual objects that allows for new interactions through gesture and voice.
- What is mixed reality?
- What are the enterprise and consumer applications?
- What are the current limitations and challenges of the technology?
- How are mixed reality applications being used to improve the workplace?
- Is mixed reality ethically neutral?
- Ann Nolan - COO & Co-founder, Snoball
- Luke Chadwick - Lead Engineer, REALABS (REA Group)
- Tim Dwyer - Associate Professor, Faculty of IT, Monash University
What is mixed reality?
In one sentence, mixed reality (MR) is where augmented reality (AR), virtual reality (AR) and the internet of things (IoT) collide or, intersect.
MR is the middle area between the real environment (reality) and the virtual environment. It consists of VR, which exists in an artificial space, and AR, which is where the virtual augments the real. It involves taking digital objects – visualisations, virtualisations – and brings them into the physical world, giving digital objects physical characteristics.
MR is more advanced than VR as it combines the use of several types of technologies, such as sensors, advanced optics and computing power. It is also less obtrusive than virtual reality as it allows its users to interact with tangible objects in a real environment, utilising overlaid augmented, digital content to create realistic scenarios.
What are the enterprise and consumer applications?
Some analysts predict the value of the virtual market – of which MR technology is a part of – will be worth US$28 billion by 2020, however it’s not quite there yet. And that’s largely because outside of gaming, consumer-facing applications for MR are limited. With more universally appealing content needed, immersive narrative-based storytelling has been flagged as an area that will lead to more users. User generated content will also push MR consumer applications, with a very basic example of this being the overlays used by Snapchat users.
The enterprise applications of MR are further ahead than consumer applications, with many industries exploring use cases and pilot applications. For example, highly realistic training can be conducted for industries where dangerous situations are regular occurrence. Utilising MR technology, resource companies (oil, gas etc.) can test safety and compliance protocols and improve procedural execution in the event of an emergency safety incident. Training and testing scenarios can also be run in the digital world, creating a realistic digital copy of a set of circumstance that couldn’t otherwise be safely replicated.
The embodied interaction MR offers i.e. physical embodiment of virtual interaction, is currently being utilised in the fields of architecture, design and user experience (UX), where virtual environments can be easily seen, interacted with and explored.
What are the current limitations and challenges of mixed reality technology?
While many at the forefront of MR technology see it as the next evolution in computing and productivity, it’s not without its challenges.
At this stage in its development, MR technology is hamstrung more so by its hardware than its software. With MR headsets suffering a similar fate to VR headsets in that the technology is cumbersome and ugly. As well as becoming uncomfortable to use after long periods – particularly for those not accustomed to wearing glass. Then there’s the issue of being tethered i.e. to a computer, which limits the tech’s usage to indoors. VR and MR technology is advancing rapidly, cutting down on latency – the time between when you move your head and when the virtual picture adjusts – however, some users reporting feeling motion sickness after using VR and MR headsets for only a short period of time.
At AU$4,369 (incl. GST), the Microsoft HoloLens ‘Development Edition’ isn’t exactly a stocking filler. And it is this price-point that’s hindering mass market appeal, keeping consumer level adoption and usage rates low. Though the price of this technology does continue to drop, HTC’s Vive can be purchased for only US$798, more universally appealing content on these platforms is needed to attract more than hard-core gamers, which is where most of the consumer-facing MR is being used.
How are mixed reality applications being used to improve the workplace?
Among the numerous and evolving applications for MR, two specific applications are emerging as game changers: training and data visualisation.
Typical training methods in high risk workplaces, such as medical fields or engineering and industrial machinery, are usually either:
- Costly to administer
- Not a true reflection of what a trainee will experience ‘in the field’, or
- Obstructed by user/subject bias.
For mass casualty injury training in particular, having the ability for medical and first responder trainees to experience a near real environment, and have the ability to make mistakes – without embarrassment or real-life repercussions – gives them experience previously not available until they actually applied learned skills.
Having the ability for ‘in the field’ usage allows for a supervisor to see the experience as the trainee sees it and provide real-time advice, without having to be on-site, which can free up significant human resources. Restarting training scenarios in these aforementioned fields can also be time-consuming, with MR technology you can simply restart a training exercise with the click of a button.
MR is a game changer for data visualisation because it allows data to be more easily perceived, manipulated and interacted with. For example, MR could be the death of the doctor’s clipboard (medical charts). As a doctor, stepping into a patient’s room and having access to up to date patient information displayed in front of you – via an MR headset – can improve processes markedly. Even better, wearable MR technology can allow a surgeon to access their patient’s medical records mid-surgery.
This kind of data visualisation and interpretation is called situated analytics and it has applications for workplaces where workers may not have their hands free – or clean enough to touch a screen – and need to access information quickly; workers can have a hands-free physical interaction with data via VR, AR and MR technology.
Is mixed reality ethical neutral? If not, what are the ethical considerations?
Ethical challenges for emerging technology don’t just stop at MR, however one unique challenge raised is how children will interpret these mixed reality scenarios. With real environments colliding with virtual environments, there is the very real potential for children to have difficulty distinguishing between the real and virtual. And with the propensity for advertising to not only infiltrate into this space, but become significantly more obtrusive, how long before we see Blade Runner/Altered Carbon-style holographic advertising jumping out at us on the street? Will a unique code of ethics or advertising guidelines specific to the technology have to be created, which define how immersive and obtrusive advertising can be?