This article is designed to be the second part of a short series offering personal thoughts on the broad state of virtual reality (VR) and augmented reality (AR, together with mixed reality, or MR) as they appear to stand at the end of 2018, and where they might be going over the course of the next few years.
In doing so, I’m not attempting to set myself up as any kind of “expert” or offer predictions per se; I’ve simply been gorging myself on a wide range of articles and reports on VR and AR/MR over the last few weeks to make sure I’m caught up on things. In part one, I covered VR; This part therefore examines AR/MR, with an emphasis on headset / eye wear, as it is these tools that particularly interest me.
Compared to VR, AR/MR has been much more a slow burner in terms of press interest. The reason for this is simple: outside of a few headliners like the original Google Glass, Microsoft’s HoloLens and, most recently, Magic Leap One, AR/MR eye wear hasn’t really caught the media’s attention. However, in assessing the state of the VR and AR/MR markets over the next 3 years, SuperData predicts something of a rapid rise in AR/MR adoption, which could see the technology generate revenues very slightly in excess of those predicated by SuperData for VR by the start of 2022.
Even allowing for these figures including smartphone AR applications, this forecast might seem optimistic, but there are reasonable grounds to suggest they are not beyond the realm of possibility – if, perhaps a slightly holistic view is taken. I say this for a number of reasons: the increasing use of AR/MR in a range of workplace / service environments; the release of development platforms for AR on smartphones and mobile devices; and availability / development of new headsets; although there are some caveats.
I’d like to examine these ideas in turn, starting with adaptation of AR/MR in enterprise-type environments. In doing so, I’m limiting myself to briefly covering just three examples: Google’s Glass Enterprise Edition, Microsoft’s HoloLens and a company called Osterhout Design Group (ODG).
Using the basic Google Glass concept (2013-2015) Glass Enterprise Edition re-lunched in mid-2017 with 50 US companies using it in engineering, training and services including GE Aviation, Boeing, Volkswagen, AECO, and DHL, and with a range of healthcare uses, including Augmedix and Brain Power (see Google Glass: The Comeback?, July 2017 for more).
Microsoft’s HoloLens has been similar adopted by a range of companies including Volvo Cars, Japan Airlines, BlueScope Buildings and Trimble (architecture and building design), Autodesk, together with widespread adoption in healthcare from training through to major aspects of surgery in hospitals around the world. Most recently, the US Army has given Microsoft US $480 million to develop the HoloLens for troop training and combat missions, while NASA utilises it both on the International Space Station (Project Sidekick) and as a mission / prototyping visualisation tool (projects OnSight and ProtoSpace).
Osterhout Design Group (ODG) – a company that potentially help Microsoft develop the HoloLens when they sold 81 patents related to AR and head-worn computers to the software giant for US $150 million in 2014. Have released a family of AR glasses, the R-7 and R-7HL (“hazardous locations”) specifically designed for use across business and industrial applications, providing heads-up information displays and overlays. In 2017, ODG launched the R-8 and R-9 glasses, utilising Qualcomm’s more powerful Snapdragon 835, with R-8 intended to start bridging the gap between “enterprise” and consumer use.
There are other examples of AR headset use in business (and entertainment) to be sure, but I hope the above are enough to make the point. Highlighting the use of AR systems in the workplace is important (as it is with VR – see part 1 of this series) because familiarity with them in the workplace could help spur people’s willingness to bring it into the home as affordable consumer systems start to appear, because: because a) they have experienced it within their workplace and have seen it benefit them; b) the hardware involved is (more-or-less) the “same” as the hardware they are buying (familiarly encourages both trust and experimentation).
I haven’t written too much about “consumer” virtual reality and / or augmented reality during 2018, primarily because this past year has been rather quiescent when compared to 2017 and earlier, so outside of one or two events, there hasn’t been that much I’ve been prompted to write about. As such, and as we pass from 2018 to 2019, it seems a good time to take a broad look at both and where they might be going, at least from a purely armchair perspective.
In doing so, I’m not attempting to set myself up as any kind of “expert” or offer predictions per se; I’ve simply been gorging myself on a wide range of articles and reports on AR, VR and mixed reality over the last few weeks to catch up on everything, and with this article I’ll focus on virtual reality.
(Note that in writing this article, I’m deliberately ignoring two products that involve VR: Microsoft Mixed Reality and Apple’s rumoured AR / VR system. The former, because Microsoft appears to be playing a much longer game, and it is unclear how MMR will impact markets down the road; the latter because it’s unclear how Apple’s product will mix AR and VR, it’s overall capabilities, price point or precise nature.)
Consumer focused virtual reality has always had a hard mountain to climb. From the start, predictions of its growth verged on the ridiculous. At the end of 2015, for example, TrendForce claimed sales of VR hardware, software and services would hit US $70 billion by 2020, a figure that, at the time tended to be taken for granted despite the fact that when it was made, the consumer versions of the Oculus Rift and HTC Vive hadn’t even started shipping. Nor were TrendForce alone in the hyping.
Obviously, VR hasn’t achieved anything like this kind of volume, but it is growing. In 2017, for example, total VR hardware and software sales reached US $2.8 billion, three years ahead of the time frame IHS Markit (one of the more reserved analytics companies looking at VR in late 2015) predicted. In 2018, this increased to US $3.3 billion; a relatively modest growth, but not unexpected given that outside of the Oculus Go, there haven’t been any major releases of VR headsets. This modest growth in sales, coupled with the lack of exciting new hardware releases has perhaps lead to more negativity around VR being voiced than previous years. However, 2019 could be the start of a “turnaround” for VR.
As it is, SuperData, which specialises in analysing the computer and gaming sectors, predicts that the VR market will double total revenues to US $9.6 billion in 2019. They further suggest revenues could grow to US $19.0 billion by the end of 2021. These might again sound like inflated figures – particularly the idea of a five-fold revenue increase in just three years, but there are actually two or three reasons to suggest why 2019 could well see significant growth in revenue for VR, and which will see it continue to trend upwards at a rate somewhat faster than seen thus far.
Up until the arrival of the Oculus Go earlier in 2018, consumer VR hardware had been more-or-less split into three areas: high-end tethered systems requiring upmarket PCs to power them; units dependent on the use of smartphones for a more limited immersive experience, and what might be termed a purely games oriented solution in the Sony Playstation VR. As such, all have been somewhat limited in their appeal / reach.
However, in 2018 the Oculus Go arrived, and in 2019 it is set to be joined by the Oculus Quest and the Vive Focus. The significance of these three units is that they are entirely self-contained and provide an immediate VR experience right out-of-the-box. No need to hook up a heavyweight PC (possibly at added expense) for the heavy-lifting, or to have a suitable smartphone to provide the visuals.
While both the Quest (shipping in 2019) and the Focus (currently only available in China) have yet to become globally available, their potential impact might be seen in the positive response the Go generated at launch, as noted by SuperData:
Oculus Go is part of an important movement. Facebook sold more units of the standalone headset in its launch quarter than they did the Oculus Rift in the entire first half of 2017. Its price and convenience are proving to be selling points.
– Stephanie Llamas, SuperData Head of XR data research
What is particularly interesting about the response is that it has not been limited to purely “home” use. While the Go is marketed as an “entertainment” headset, it has already been seen as a means of expanding VR’s use within enterprise markets. Take Walmart as an example.
Magic Leap has had its share of ups-and-downs over the past few years. Founded by tech wizard Romy Abovitz, the super-secret company has been at the centre of hype, speculation and doubt. Much of the hype has been spun by the company itself, much of the doubt has been driven by reports of friction in the company, issues with the technology, and so on.
Now, a year after scepticism around the company overtook the hype, Magic Leap has announced the availability of its first product: the Billed as the Magic Leap One Creator Edition augmented (or mixed) reality system, the unit comprises three parts:
Lightwear: a headset utilizing the company’s “Digital Lightfield” display technology with multiple integrated sensors to gather spatial information.
Lightpack: a circular belt-worn hip pack that contains the computer powering the headset via a tether.
Control: a hand-held controller that can be tracked in space that helps users navigate menu selections, etc.
As an augmented / mixed reality system Magic Leap One is designed to blur the divide between the digital and the real, with the company promoting a series of potential use-cases for it, including: web browsing and shopping, working on multiple virtual monitors, social telepresence, theme park “rides” and experiences, and gaming.
The system is somewhat removed from some of the hype built-up around Magic Leap’s initial designs – which tended to suggest something far more glasses-like.
There’s no doubt the headset is a lot bulkier than might have been imagined from past descriptions, and while nowhere near as bulky as a VR headset, it leaves a lot to be desired in the ergonomics department, particularly when compared to the likes of AR headsets like Google’s Glass Enterprise Edition or AR systems using Qualcomm’s snapdragon processors. My own impression on seeing the Magic Leap One images is that the headset looks sci-fi bug-eyed – almost sinister – and the size of the lenses has me wondering about effective field-of-view.
The Lightpack has also come in for critique, with some tech journos calling it “large” or “bulky”. been called “large” by some in the tech press, it’s actually about the same size as Walkman CD players people used to happily clip onto their belts and wear.
The Control has a trackpad and six degrees of freedom (6DOF) tracking, and some six option buttons.
Other than that – details are currently light right now. There are no technical specifications or pricing. However, and in fairness, Abovitz does refer to the announcement as “a small reveal“, rather than any kind of pre-release notification. Instead, interested parties (defined as developers, reporters and the curious) can register their wish to learn more by supplying their e-mail details via a form as the bottom of the Magic Leap home page.
So far, Magic Leap has demonstrated various iterations of their equipment to assorted people from the technology and entertainments industries. All seem to have been thoroughly impressed – although sworn to secrecy – which has been frustrating for those trying to figure out exactly what the company has got. This approach actually continues with this pre-announcement about Magic Leap One – Rolling Stone magazine has an extensive article about Magic Leap in Glixel – but the use of an NDA prevents much in the way of really solid facts around the technology from being revealed, while descriptions of environments are sans images.
There are, however, some intriguing little pieces of information within the article – such as this ability to generate very life-like characters, which Brian Crecente, writing for Glixel, suggests could become a kind of virtual assistant for those using the Magic Leap:
She walked up to me, stopping a few feet away, to stand nearby. The level of detail was impressive, though I wouldn’t mistake her for a real person, there was something about her luminescence, her design, that gave her away. While she didn’t talk or react to what I was saying, she has the ability to [do so] … I noticed that when I moved or looked around, her eyes tracked mine. The cameras inside the Lightwear was feeding her data so she could maintain eye contact. It was a little unnerving and I found myself breaking eye contact eventually, to avoid being rude.
One day, this human construct will be your Apple Siri, Amazon Alexa, OK Google, but she won’t just be a disembodied voice, she will walk with you, look to you, deliver AI-powered, embodied assistance.
Which sounds very sci-fi-ish, raising the idea of virtual tour guides and suchlike – as well as the question of whether or not we’ll have to cross the uncanny valley with AR as well as (possibly) VR.
I’m somewhat of the belief that AR / MR has the potential to be far more ubiquitous that VR, and garner a much larger, multi-use audience. The likes of Glass Enterprise and several of snapdragon headsets demonstrate considerable interest within healthcare, engineering and retail. The very nature of the technology means it can be integrated far more easily into our everyday lives and work than VR allows. That said, where and if Magic Leap fits into all of this remains as murky as ever. Perhaps the upcoming “Creator Portal”, promised for “early 2018”, coupled with a lifting of the restrictions concerning direct reporting on the system will do more to answer questions.
This month has seen some interesting press pieces popping up concerning VR and Sansar since the opening of the Creator Beta. However, three in particular have so far caught my eye as they appeared, as they offer interesting perspectives and discussion points both on the Lab’s new platform and on VR and AR as a whole.
The first – and most recent, is Barely into Beta, Sansar is making social VR look good, by Alice Bonasio, which appeared in The Next Web on August 18th. The title caused some to question Sansar’s social capabilities, but the article itself was more about Sansar’s overall status and development, rather than zeroing directly into the medium of “social VR” per se. In this respect, it opens by clearly underlining the platform is still in its early days, and there is still much to be done, using a quote from Peter Gray, the Lab’s Director of Global Communications, to do so:
We wanted to make Sansar available to everyone as early as possible, and there are still a lot of features and capabilities that we’re excited to add to the platform soon, as well as many improvements to the current featureset.
From here, Ms Bonasio makes the point that despite the lack of features and capabilities which will be needed to fulfil on its promise of being a social hub, it already looks good and offers a lot to see, much of which points to the platform’s potential.
The piece also delves into some of the technical and economic factors which set Sansar apart: such as Linden Lab’s partnerships with IKinema and Speech Graphics. The former is key to the Sansar avatars utilising Inverse Kinematics in an advanced way, and which are and will play a key role in the Sansar avatar’s development. The latter is key to synchronising facial animations automatically to match speech patterns, a capability key to many of the social interactions Linden Lab hope will be occurring within Sansar.
The article also touches on some of the key differences between Sansar and Second Life, the ability Linden Lab has to take fourteen years of running a virtual world to help shape the philosophy and approach it takes with Sansar. Passing – but important – mention is made of the Lab’s ability to self-finance Sansar; given the topsy-turvy situation with Altspace VR (which may have been saved from having to close), this is an important fact to keep in mind.
As noted above, the piece has received some feedback questioning the “social” element of Sansar at it stands at present, which given the broader thrust of the article might be considered a little out-of-context. However, it is fair to say that right now Sansar does currently lack elements which could be regarded as essential to supporting larger-scale social activities. Similarly, while social interactions are possible – as demonstrated through the daily meet-ups held “in-worlds” – it’s also fair to say these can be confusing and limiting for some. For example, undisciplined voice chat can mean that that multiple conversations in a single locale can overlay one another and become confusing to those not used to voice chat.
Hopefully these issues will be addressed, along with the provision of other social elements, and I’ll doubtless have more to say on them myself in the future 🙂 . In the meantime, this article provides a good summation of Sansar for the curious / those wishing to catch-up on things.
I’ll say up-front that I’m one of the non-believers that VR will become ubiquitous for business-style conference calls for a number of reasons, and its fair to say that Samantha Cole does a balanced job of presenting both sides of the argument – whilst also offering side pointers to those areas where VR is already showing benefits (and which I’d suggest Sansar could leverage).
Much has been made of VR’s abilities to add body language, hand movements, eye movement and contact – all vital elements in adding subliminal feedback / context to our day-to-day, face-to-face interactions to one another – to give more depth and meaning to tele- and video-style conferencing. In doing so, the likes of the telephone and “traditional” means of this type of conferencing have been somewhat “demonised”. Emphasis is laid on things like network latency, or the extra mental effort involved in reading into people’s words when you can only hear their voice or see their head / shoulders, as “limiting” such interactions.
But the truth is, we’ve been using the telephone for decades as a business tool. It’s fast and convenient, and as adults, we’re all pretty adept on picking-up on vocal nuances. We’re also, in a business context, far more prepared to communicate directly with colleagues; if there is something worrying / irksome within a work environment / business project, most of us are pretty willing to make thought known, be they over the ‘phone, face-to-face or via e-mail. So even with the faster, lighter, better VR technology we’re promised will be coming down the pipe, is it really any kind of “killer app” for business conferencing?
Eric Boyd, a professor of marketing at James Madison University points to emerging trends within the workplace as a whole being more a deciding factor here. Many companies have experimented with remote / home working over the past 2 or so decades, and the pendulum tends to swing back and forth. Right now, as the article points out, one of the first to enter the arena of remote working, IBM, is currently backing away from it. Thus, if working practices remain centralised, it’s hard to see VR overturning technologies already in place and supported by existing corporate infrastructure, no matter what the perceptions of their “limitations”. But for those organisations continuing to embrace remote working, VR could become a useful meeting tool.
Certainly there would seem to be far better uses VR could be put towards within a business environment: prototyping, training, simulations, and so on, which seem far more likely to drive its adoption by business and industry far more than the humble conference call. In this, Cole’s pointing to VR’s potential in training and simulation and in architecture is very salient; these are very much markets well suited to VR / AR / MR – perhaps more so that conference calls.
While Sansar is only mentioned in passing (together with the downs and ups of AltspaceVR), the article is interesting as it encompasses the viewpoint of a company investing in VR and AR start-ups with funding in the US $100,000-500,000 range – which is small when compared to the likes of the big players, but has allowed the company to bask some significant start-ups, including STRIVR, who are in the VR training a simulation field mentioned above.
The article opens which a rapid-fire overview of the VR / AR market – including its niche status at present, which could be said to be largely down to the limitations of the current hardware (or lack thereof in AR’s case, although that is beginning to change) rather than anything else. However, the meat of the piece is where Mahajan sees the technologies going over the next several years.
What’s interesting here is that within Presence Capital, they are moving away from consumer-focused VR endeavours and more towards business and business-to-business (B2B) / enterprise VR applications as well as for AR; he points to the likes of AppliedVR and their development of an immersive platform to help comfort patients undergoing painful procedures, and also underlines VR’s application in training.
This year’s swing towards AR is also examined: Google, Apple and Facebook are all looking to develop AR platforms, and the discussion looks at these and at the questions of standards, formats, and enabling technologies. In this, Mahajan points somewhat towards the eventual merger of AR and VR to produce Mixed Reality, indirectly pointing to how AR – augmented reality – could actually become an enabler of VR (something the likes of Qualcomm are working towards with Android and their snapdragon chipset), simply because it will allow both to coexists as tools people can switch between according to needs.
All three article make for interesting reads, presenting a broad range of perspectives not just on Sansar (in the case of Alice Bonasio’s piece) but on VR and AR as whole.
Five years ago, Google Glass leapt (literally – the product launch included a team of skydiving Glass wearers) into the public consciousness. At the time it went through a pretty rapid-fire hype cycle: from the kit everybody would want (with no clear understanding of what it was really for), to the reality of a buggy, poorly implemented system to over-hyped fears of privacy invasion and a slew of resultant bannings of the hardware from all manner of places.
So rapid was the rise and fall of Google’s premature launch into the world of augmented reality (from arrival to apparent death in three years) that many in the media wrote it off as the butt of jokes and pointed to it as a reason why AR and VR could well be fads.
Only, as Google revealed on Tuesday, July 18th, and as superbly reported on in depth by Steven Levy for Wired’s, Backchannel, Google Glass never actually died. Google just did the sensible thing – admitted they’d got their original vision for the product wrong, quietly turned the page on Glass as a consumer product and focused on developing the technology into something people actually wanted, and were themselves working to create using Glass.
These “people” were companies in the manufacturing and service sectors who had seen the potential for the headset system and had started buying units and developing software to use with them. Companies like General Electric, GE Aviation, Volkswagen, Boeing, DHL, agricultural equipment manufacturer AECO (featured in the Wired piece) and healthcare system provider Augmedix all got involved with Google Glass. What’s more, Google noticed, and started re-aligning the headset’s development. Hence why in January 2015, the consumer version of the headset was brought to an end with the comment posted the Glass website: “Thanks for exploring with us”— The journey doesn’t end here.”
Alphabet X, the R&D arm of Google’s parent company, Alphabet, took over the development of Glass, working closely with the companies putting it to work. Although based on the 2013 Google Glass Explorer Edition chassis, the new Glass Enterprise Edition (“Google” has been entirely dropped from the name) is an almost new headset, featuring:
Improved electronics – camera (complete with a red recording light to let others know when the camera in being used), wifi, processor.
Improved battery life and recharging.
A removable”Glass Pod” containing all the system’s electronics, which can be mounted on safety glasses, allowing Glass to be used in environments where eye protection is required, or used with prescription glasses.
In his piece, Steven Levy dives into some of the areas where Glass is already being used to great effect by several of the organisations mentioned above, and the list of blue-collar and service environments where Glass is being used is interesting and diverse, offering a glimpse of the potential for AR.
Augmedix, for example has been pioneering the use of Glass with a number of healthcare organisations to help improve doctor / patient interactions. The headset is use to access medical data and help keep the doctor away from a computer screen, allowing them to decrease the amount of time per consultation they are spending working on a computer whilst increasing the amount of face-to-face direct interaction with patients. Further, thanks to the use of an unseen “scribe” – a medical student or trained medical transcriber – who might be down the hall, in another state or even in another country, the doctor can dictate updates to the patient’s records, provide information on prescriptions, etc., removing the 2-3 hours a day they otherwise need to spend managing the records on their own – again allowing more time to be spent with those in their care.
The other point to note here is that this is not the announcement of some beta programme; it’s the launch of an actual product by Alphabet. “This isn’t an experiment,” Jay Kothari, Glass Project Lead, said. “It was an experiment three years ago. Now we are in full-on production with our customers and with our partners.” Not bad for a product written off as dead just 24 months ago.
The announcement also means that the companies that have helped developed the software etc., to run alongside of Glass – as with Augmedix – are now free to roll Glass out as they need, and to start marketing their products developed to work alongside Glass. Google also have plans of their own to further extend Glass’ reach as an enterprise tool – although they remain silent as to whether the consumer product will be resurrected.
The announcement about Glass points to 2017 as possibly being the year in which AR / MR starts on its rise to practical prominence. It joins Microsoft’s HoloLens as an enterprise tool, while Windows 10 offers a platform for MR development (and Microsoft are working with hardware manufacturers to provide consumer focused headsets). Elsewhere, Qualcomm – as I recently reported – is leading the charge with Android-based enterprise and consumer AR / MR headsets. It’ll be interesting to see where all the leads, both in the work place and – in time – at home.
Augmented reality headset maker, CastAR (formerly Technical Illusions), and which I’ve been following and reporting on in this blog, has apparently closed its doors.
The company first came to prominence when the system – designed to use a glasses-like headset to project holographic images onto a retro-reflective surface users could then interact with via a hand-held controller – was shown at the 2013 Maker Faire in New York.
At that time, the idea was very much fledgling and more spirit gum, soldering, tape and wires than it was a commercial venture – but that was enough to convince the pair behind the system, Jeri Ellsworth and Rick Johnson they had a potential product on their hands, so they went ahead with a Kickstarter campaign to raise US $400,000 to start development in earnest – and ended up raising over a million.
The system, called CastAR due to its primary function of projecting images onto that retro-reflective surface, actually came about by accident. In 2012, Ellsworth was working on another project at Gabe Newall’s Valve when she accidentally found she could create holographic-like images on the surface.
The potential of the idea excited her and (then) fellow co-worker Johnson, so they started delving into the idea. Then, in February 2013, they were among a group of staff let go by the company – and in a generous move, Newall allowed them to take the IP for the CastAR system with them, even though almost the entire development up to that point had been made on Valve’s time and with Valve’s resources.
From here, the story does suffer the hiccups. The Kickstarter raised US $1 million, enough to fund development of an initial headset system, but it was not entirely what Ellsworth and Johnson were hoping it would be. There were delays, funding seemed (from the outside) to be slow in coming in and delivery dates for the initial Kickstarter headset got pushed back, although there was sufficient for the company to establish operations, hire a CEO (initially David Henkel-Wallace), refine the headset design, develop games to run on it.
But the company kept doing the rounds of VR / AR shows and the like, garnering publicity, generating interest and towards the end 2015, secured US $15 million in funding. The majority of this came from Playground Global, co-founded by Andy Rubin of Android Inc. fame, and Rubin persuaded Ellsworth and Johnson to back to basics and design the system they wanted.
As a result, in 2016, the company announced a significant change in direction. The US $1 million raised via the Kickstarter was refunded, together with a promise all backers would receive a “consumer” version of the headset, CastAR hired talent to open its own mixed reality studio in Salt Lake City and acquired entire Eat Sleep Play development team, responsible for the Twisted Metal series and God of War. All of this was done with the aim of developing a complete consumer package – headset, controllers, game surfaces and games – which would be low-cost and playable “right out of the box”. 2016 saw LucasArts chief Darrell Rodriguez take over as CEO, with former Disney executive Steve Parkis as its president and chief operating officer.
Now, according to an article appearing in Polygon on Monday, June 26th, and since widely circulated in the tech media, CastAR has closed its doors with up to 70 people being laid off. There has been no official statement on the matter from either CastAR – the corporate website continues to reference a consumer product launch in 2017, although it doesn’t appear to have been updated since around the start of the year – nor Playground Global, despite attempts by a number of outlets to secure a comment. However, the Polygon piece suggests the reason for the closure is Playground Global’s refusal to provide further funding for the venture after CastAR failed to obtain investment from other sources.
Following the story breaking, Polygon later updated their article to reflect sources stating that a small team has been retained by CastAR to oversee attempts to sell the company’s IP. But again, there has yet to be an official statement from CastAR.
This might be seen as a blow to the fortunes of AR. However, as innovative as CastAR was (and as much as I found their approach fascinating), the system took a markedly different approach to AR / MR than is the case with the likes of Qualcomm (see here) and others, simply by its reliance on a retro-reflective surface. While the latter is well suited to gaming, and the company tried to suggest it could have practical applications through their promotional videos, it still might have been seen as a limiting factor in the system’s broader appeal.