Thoughts on VR and AR, part 2: AR, MR and beyond


This article is designed to be the second part of a short series offering personal thoughts on the broad state of virtual reality (VR) and augmented reality (AR, together with mixed reality, or MR) as they appear to stand at the end of 2018, and where they might be going over the course of the next few years.

In doing so, I’m not attempting to set myself up as any kind of “expert” or offer predictions per se; I’ve simply been gorging myself on a wide range of articles and reports on VR and AR/MR over the last few weeks to make sure I’m caught up on things. In part one, I covered VR; This part therefore examines AR/MR, with an emphasis on headset / eye wear, as it is these tools that particularly interest me.

Compared to VR, AR/MR has been much more a slow burner in terms of press interest. The reason for this is simple: outside of a few headliners like the original Google Glass, Microsoft’s HoloLens and, most recently, Magic Leap One, AR/MR eye wear hasn’t really caught the media’s attention. However, in assessing the state of the VR and AR/MR markets over the next 3 years, SuperData predicts something of a rapid rise in AR/MR adoption, which could see the technology generate revenues very slightly in excess of those predicated by SuperData for VR by the start of 2022.

AR / MR revenue trends, 2018-2021. Credit; SuperData

Even allowing for these figures including smartphone AR applications, this forecast might seem optimistic, but there are reasonable grounds to suggest they are not beyond the realm of possibility – if, perhaps a slightly holistic view is taken. I say this for a number of reasons: the increasing use of AR/MR in a range of workplace  / service environments; the release of development platforms for AR on smartphones and mobile devices; and availability / development of new headsets; although there are some caveats.

I’d like to examine these ideas in turn, starting with adaptation of AR/MR in enterprise-type environments. In doing so, I’m limiting myself to briefly covering just three examples: Google’s Glass Enterprise Edition, Microsoft’s HoloLens and a company called Osterhout Design Group (ODG).

  • Using the basic Google Glass concept (2013-2015) Glass Enterprise Edition re-lunched in mid-2017 with 50 US companies using it in engineering, training and services including GE Aviation, Boeing, Volkswagen, AECO, and DHL, and with a range of healthcare uses, including Augmedix and Brain Power (see Google Glass: The Comeback?, July 2017 for more).

  • Microsoft’s HoloLens has been similar adopted by a range of companies including Volvo Cars, Japan Airlines, BlueScope Buildings and Trimble (architecture and building design), Autodesk, together with widespread adoption in healthcare from training through to major aspects of surgery in hospitals around the world. Most recently, the US Army has given Microsoft US $480 million to develop the HoloLens for troop training and combat missions, while NASA utilises it both on the International Space Station (Project Sidekick) and as a mission / prototyping visualisation tool (projects OnSight and ProtoSpace).

  • Osterhout Design Group (ODG) – a company that potentially help Microsoft develop the HoloLens when they sold 81 patents related to AR and head-worn computers to the software giant for US $150 million in 2014. Have released a family of AR glasses, the R-7 and R-7HL (“hazardous locations”) specifically designed for use across business and industrial applications, providing heads-up information displays and overlays. In 2017, ODG launched the R-8 and R-9 glasses, utilising Qualcomm’s more powerful Snapdragon 835, with R-8 intended to start bridging the gap between “enterprise” and consumer use.
The ODG R-8 and R-9 headsets, launched at CES 2017. Credit: Engadget

There are other examples of AR headset use in business (and entertainment) to be sure, but I hope the above are enough to make the point. Highlighting the use of AR systems in the workplace is important (as it is with VR – see part 1 of this series) because familiarity with them in the workplace could help spur people’s  willingness to bring it into the home as affordable consumer systems start to appear, because: because a) they have experienced it within their workplace and have seen it benefit them; b) the hardware involved is (more-or-less) the “same” as the hardware they are buying (familiarly encourages both trust and experimentation).

A further boost for AR / MR could come via smartphones and tablets. We’ve already seen how AR games can captivate attention through the likes of Pokémon Go, while a number of companies (Ikea and Amazon being two brand leaders) have been experimenting with “retail experiences” for mobile devices.

Thanks to the 2017 launch of Apple’s ARKit for iOS and Google’s ARCore, AR apps could become increasingly visible: when ARCore launched for example, it was only for a limited range of high-end Android devices. But by December 2018, it was estimated some 250 million Android devices can run ARCore apps – with the number rising.

These type of apps will inevitably help boost the use of AR eye wear as more systems become available. As it is, AR glasses have the potential to become as ubiquitous in our daily lives as the smartphone, because there is so much we do via our smartphones – communications, entertainment, socialising, taking photos, navigation, and so on, that can naturally be translated to a fully heads-up, hands-free and unobtrusive use supplied by a pair of decent-looking glasses. So the availability of smartphone-based AR apps could encourage people to think more about making a move to using a headset.

BMW has experimented with AR heads-up manuals to help guide people through aspects of car maintenance. Crdit: BMW (screen cap)

Several companies are already starting down the route. For example, ODG have launched their R-8, noted above, while Optinvent has launched the nifty (of a little ugly) Ora-2, and Vuzix are making major waves with their Blade glasses. All of these systems can be used as either stand-alone glasses utilising heads-up displays (the Blade having the advantage of fully built-in Alexa support) or can be linked to an existing smartphone to put apps and capabilities directly in front of the user’s eye(s).

Currently, perhaps the biggest issue with these types of glasses is not so much looks – the Vuzix might almost be taken as a pair of Oakley sunglasses, for example – but a combination of price and utility. Vuzix, for example, is promoting  the Blade more as a smartphone companion than a wearable computer (although the Alexa integration is highlighted) – but are people really going to pay US $1,000 for something that “just” moves their smartphone screen in front of their eye, or are they just going to keep on doing what they’re doing with their ‘phone and forget the added expense?

AR is entering niche consumer markets such as cycling and running, with products like the Everysight Raptor (above, US $649) and Solos (US $499) glasses through to clip-on information systems like the Garmin Varia Vision (US $399)

In terms of utility, another possible drawback with these kinds of glasses is they don’t actually offer the kind of MR immersion offered by the attention-grabbing HoloLens and Magic Leap One (a system that I’ve thus far intentionally ignored, as it seems to be having problems making a mark for itself thus far), which may also limit appeal. So a bridge between the two is needed.

Well, this is also coming – but it may be another few years before we see it reach “affordable” pricing. Chinese start-up Nreal, for example, demonstrated a set of rather smart-looking glasses at the 2019 Consumer Electronics Show.

The Nreal system from China, powered by the latest Qualcomm Snapdragon 845 with both 6DoF and SLAM, slated to go on sale at the end of 2019. Images: The Verge / Nick Statt

These offer both heads-up AR displays / overlays and immersive holographic MR with a wider field of view than either the HoloLens or Magic Leap One (although the latter has some way to go before it is market-ready). Ultra-light, the system takes a leaf from Magic Leap by incorporating the processor in a belt-clip box, and utilises a small hand controller. The system’s stated retail price will be US $1,000, still on the high side for a consumer product with broad reach.

A January 2019 Microsoft patient provides insight into their direction for smartglasses (and which also appear to use tech that originated with ODG). These also appear to combine heads-up capabilities with the holographic MR within a sunglasses-like form factor. The patent outlines a wide range of use-cases and supporting technology; as such the potential for a consumer headset, its price-point and possible availability is hard to judge.

Diagrams from Microsoft patent 2019/20190025587 for an AR / MR headset chassis / system, showing the glasses (top left and right) and examples of the heads-up AR overlay and MR projection (a holographic keyboard onto the wearer’s hand). Credit: Microsoft

Further down the road – say in 5 or so years – there’s a good chance we’ll see technology reach a point where AR/MR and VR will significantly merge, with glasses / headsets capable of doing all three. This is the vision Qualcomm are developing in what they (were the first to?) term XR (“eXtended Reality” see Enter the Snapdragon, June 2017). In 2018, Qualcomm launched its XR1 development platform, the first step in this vision with Oculus, HTC Vive, Pico (VR), Vuzix and Microsoft among others already on-board.

We see a journey where AR glasses and VR HMDs are going to evolve year over year … we’ll start seeing converged products that can do both. In fact, we already see examples of AR glasses that enable some VR 360 experiences. Similarly, with VR HMDs, we’re starting to see some integrations of AR experiences with pass-through camera feeds.

– Hugo Swart, Senior Director of Product Management, Qualcomm Technologies, October 2018

There are multiple improvements that need to be made for this convergence to occur: display quality, optics, processing capabilities, connectivity, software (e.g. eye tracking, latency, etc), and ultimately, affordability. However, the fundamental technologies for AR/MR and VR are similar (motion tracking, screen requirements, power constraints, etc) and even overlap, so it’s not hard to see it coming about. It’s why I tend to believe that if there is any such thing as a “killer app” for truly mass market VR, it is actually AR/MR, simply because it has the potential to slot itself neatly into our lives. So, by doing so, it could encourage a wider exploration of VR.

But any merging of VR and AR/MR is for the future. In the near-term, 2019 has already been labelled the “break-out” year for VR, much of it pinned on the arrival of the Oculus Quest (even with its relatively modest projection of just 1.3 million units sold in the year, which has me feeling 2019 won’t be any more a “break-out” year than 2018 or 2017). Certainly, 2019 will not be a break-out year for AR / MR – but it might be one in which we start to see them becoming more prominent and, in terms of software, more available to us.