Enter the Snapdragon: Qualcomm and “XR”

The ODG R9 AR headset. Credit: Osterhout Design Group

It’s no secret that when it comes to augmented reality (AR) and virtual reality (VR), I’m swayed more towards AR and “mixed reality” (MR) as potentially being the “thing” of the future. Not, as I’ve often said, that I don’t believe in VR – it will in time grow to fill various niches and requirements. Rather, I just feel that AR / MR have a much wider field of application when it comes to impacting our daily lives.

I mention this because earlier in June I read an interesting piece by Dean Takahashi, examining Qualcomm’s emerging role in what they like to call “XR” – or “eXtended Reality”, which they define as a fusion of VR, AR and MR.

Qualcomm spells out the hurdles to ‘extended reality’ glasses offers a transcript  from a chat Dean had with Tim Leland, Qualcomm’s vice president of product management, on the company’s goal and the challenges they see in bringing headset-style devices to the market.

Qualcomm, of course, is the company behind the veritable Snapdragon family of processors. In 2016, they announced their intention to make the Snapdragon 835 chipset the heart of a new range of self-contained VR and AR devices. To that end, they are about to start shipping the Snapdragon 835 VR HMD to OEMs wishing to produce Android-powered VR headsets using the chipset and Google Daydream.

More particularly – from my perspective – Qualcomm has already partnered with Osterhout Design Group (ODG), to develop a range of Snapdragon-powered AR headsets. I first became aware of the first of these units, the R7, in mid-2016. Intended to be a heads-up AR system for enterprise solutions (selling at US $2,750), it has gained a degree of traction in a number of fields – hazardous environments (oil exploration and production, chemical production and pharmaceuticals, healthcare and surgery), and has been involved in tests helping the visually impaired.

In 2017, ODG are due to release two more units – the “prosumer” R8 (around US $1,500) and the “consumer R9 (at “sub-$1,000”). Again, these are Snapdragon 835 based, and will be fully self-contained units with Android and their operating system.

It is the self-contained aspect of such headsets which Qualcomm sees as being one of the keys to the future success of “XR”.

“Any type of cable is just a non-starter,” Leyland notes. “Fans will not exist. We think there might be a niche market for glasses that maybe stream to a PC, but that’s a small part of it. The big part is everything self-contained in a mobile device. All the visual processing systems are very close to the inertial sampling systems, so everything is very fast.”

Qualcomm see “XR” systems potentially becoming a mainstay of our daily lives, fusing VR and AR into a single headset unit which can meet a variety of needs at any given time, and which can also be used as the basis for specific use-cases.

A conceptual “first responder” XR headset for fire fighters. Credit: Qualcomm

In this Qualcomm see XR units being both general purpose and specific to market sectors. The company is already looking at a concept for a “first responder” headset for fire fighters. Containing night vision capabilities and thermal imaging sensors, the headset could allow fire fighters overlay their field-of-view with floor plans of the building they are in, helping them find their way through the smoke, while the thermal sensors warn of potential hotspots and possible behind-the-door risks of backdraft – and could even guide them to people trapped in a burning location.

A simplified rendering of the kind of information the “first responder” headset might provide. Credit: Qualcomm

For more general use, Qualcomm are looking at headsets which integrate much of what they’ve developed with the likes of ODG – multiple cameras, integral motion tracking, the ability to track eye and hand movements, etc., but in a very lightweight, unobtrusive form-factor with a low price point which makes them an attractive proposition.

Not that this is going to happen overnight. A refreshing aspect of Qualcomm’s view is that they are looking at a development / adoption curve measured in at least a decade. As Leyland notes, the ability to have AR and VR heads headsets exists today, but there are hurdles to be overcome before they are as ubiquitous as the mobile phone for many of the tasks we perform today.

Some of these hurdles are being independently addressed – 5G, for example, is expected to be of huge benefit to those uses which require a lot of rendering and so are latency intolerant. Others are going to take time to progress and solve:  display requirements – the vergence and accommodation conflict, human field of view (190ox130o) etc; common illumination); motion and tracking for intuitive head, hand and eye movements; and power and thermal issues.

The technical hurdles “XR” needs to overcome. Credit: Qualcomm

Leyland doesn’t see any of these hurdles as being problematic – he just emphases that the time frame required to solve them is not going to be as compressed as some of the more bullish predictions about VR growth made in 2016 would have us believe. Instead, he points to 2020 as still being a year when numbers of shipped headset units of all types is still measured in the hundreds of millions, although he does see it growing from there.

IDC VR shipment numbers (in thousands), have been seen as a means to question the reality of the VR market

But will these systems ever reach the ubiquitousness of the smartphone? Right now, going on the shipments of VR headsets some are quick to pooh-pooh the entire mixed reality (or XR if Qualcomm prefer) ecosystem in favour of alternatives. On the surface, they would seem to be right – but on a longer-term look? I’m not so sure. Again, this is where the much-hyped smartphone analogy with VR is misleading – as Leyland points out in talking to Takahashi.

While it is true the first “genuine” smartphone as we know them today only appeared a decade ago, the fact remains that it was founded on some three decades of cellular phone development. right now, headset capabilities are roughly in the “1990s” phase of that overall curve – so there is a way to go. As such, while headsets that more closely resemble glasses / sunglasses may not necessarily become as all-pervasive as smartphones are today, there is little reason to doubt they could – if they have an intuitive ease of use – take over from handsets (and associated wearables) for a wide variety of tasks / uses.

Qualcomm isn’t alone in pursuing a convergent future of mobile VR / AR / MR capabilities. However, through Dean Takahashi’s article (and courtesy of Qualcomm’s Augmented World Expo presentation, it is good to see how level-headed is the approach being taken be tech companies to bot understand the technology , its potential and to look beyond the buzz phrases like “killer app” or order to make “XR” work.

Advertisements

A salient warning about “social VR”?

Will social VR of the kind currently being developed really be what a mass market is looking for? (image via Upload VR)

Balaji Krishnan appears to be a man on a mission: to offer a wake-up call to those engaged in the nascent world of “social VR” that the kind of future they’re chasing might not exist. He’s most notably pursuing this mission in op-ed pieces. In March he put his case Upload VR under the succinct title: In Why Social VR Probably Won’t Work the Way Social VR Developers Think, (subsequently reprinted on May 1st by PSFK.com). In April he followed it up with a more targeted piece for VentureBeat: Sorry, Zuck: AR & VR won’t replace TVs or phones.

In the first article Krishnan – the founder and CEO of Dabkick, which credits itself as developing the first “true Social VR experience“, states his case pretty clearly through the title of the Upload VR article: that social VR may not work the way most “social VR” developers – he notes Valve, High Fidelity, AltSpace VR, Linden Lab and Facebook in particular – expect.

Balaji Krishnan

This is not to say he thinks these will fail; rather than they won’t achieve the kind of mass-market prevalence we’ve seen with the likes of smartphones – the technology VR is often touted against as having the same disruptive potential.

Now, to be fair, I don’t agree with all of his points. In particular, the slow growth in the volumes of shipped headsets to date is not indicative that they won’t grow faster in the future; particularly as the technology finds its footing and the price-point computational power required for high-end systems comes down and overall quality and ergonomics of headsets improves with future generational developments. But – and here’s where I do agree with Krishnan: the hardware and the price-point aren’t the key to getting VR to appeal to a mass market.

Rather, the key to getting VR viral in the manner of smartphones is presenting it as having a convenient relevance to people – whether as a source of entertainment or social engagement or business or gaming or whatever – that’s important. And that’s actually a tough nut to crack.

The pervasiveness of smartphones is in part down to their sheer convenience and in part down to the organic way in which their capabilities have naturally grown to encompass applications and uses outside of voice communication. In trying to find the “killer app” for VR, it feels as if it is being forced down various paths in which it is unlikely to succeed in the same way as the smartphone (image via the BBC)

Take smartphones for example – as Krishnan does.That they have become a central pillar of many people’s social activities, spawning an entire ecosystem of applications and opportunities for sharing and creative experience wasn’t planned or engineered from the outset. It came about because someone realised that just as MP3 players could offer music on the go, then so could a ‘phone. And if you stuck a camera on a ‘phone, people might like to take pictures with it. It was an organic process – one which never lost sight of the ‘phone original intent: a convenient means of communicating, and built on that convenience over time until the smartphone became an indispensable part of our daily lives.

DabKick’s “social VR experience”

However you look at it, VR isn’t anywhere close to the ubiquitous nature of something like a smartphone – nor, really, can it be.  So trying to present or engineering a future where it can be is perhaps shooting wide of the mark. And really, the idea of “social VR” is another way of trying to engineer a future for VR which might not really stand up to the litmus test of what a “mass market” actually wants.

As it is, we’ve had around a decade of organic development and growth of a “digital social ecosystem”; one that offers many, many ways of engagement which are flexible enough to meet our needs wherever we are, and whatever we’re doing.  Krishnan argues that if “social VR” is to succeed, it must feed into this ecosystem, nurture it, support it and add value to it; seeking to simply “revolutionise” it isn’t enough. It must be intuitive enough to be used quickly, easily and conveniently wherever someone is and whatever they might otherwise be doing. if not, then it’s unlikely to spark people’s imaginations enough to buy into it as massively as is hoped.

So where does that leave something like Sansar? On the one hand, and as I’ve oft stated, it is pretty clear that there are markets where VR can have a significant impact. As such, if Linden Lab can hit all the desired nails on the head, then the platform could enjoy considerable success within those markets. On the other, the idea that it could become a broad-based “social” environment, outside of very specific use-cases, perhaps doesn’t stand up so well, for the reasons outlined above. Simply put; people can already undertake wide-ranging social activities through digital means, individually and collectively; simply dangling “VR” in front of them may not necessarily persuade them they need to change how they’re doing so.

Whither goes VR?

The Void, via Develop

2016 was the year of virtual hype whereas 2017 is the year of actual realities, in terms of what is achievable from a business sense related to market size, opportunities and potential revenues. Should developers or publishers get involved now? Absolutely but with an intelligent approach and realistic expectations of what these early days sales returns could be.

So considers Sam Watts, director of immersive technologies at Make Real, in a comment quoted by Jem Alexander, in the first part of his series penned for Develop looking at the hype, tech, hope, hang-ups and potentials of Virtual Reality.

Jem Alexander

The piece stands as a reasoned look beyond the hype of VR’s 2016 rebirth, thanks to the arrival of the Oculus Rift, HTC Vive, Cardboard, Daydream, Gear VR and – perhaps most importantly of all, at least in terms of sales – Playstation VR.

Yes, there has been a lot of hype, which the first year of consumer facing VR really hasn’t matched. Some have seen this as cause to deride the VR movement as a whole, relegating it to the role of “fad” – which in itself is perhaps a tad premature.

In writing this two-part series, Jem Alexander avoids both extremes and instead offers a discussion which is reasonably balanced and reasoned – and offers a perspective from many of those at the sharp end of the industry. In the first part, entitled  Where are we now, they offer an honest assessment of the market which is both positive while carrying a solid note of caution for those considering taking the VR plunge, as shown in the quote from Sam Watts, above.

There are several reasons why VR’s first year hasn’t lived up to the hype. For a start, the hardware isn’t exactly off-the-shelf, as those talking to Alexander notes. There’s plenty of room for improvements in the tech and the quality of the experience and offer it at a much lower price-point than today. Thus, taking the first year’s sale figures as being indicative of VR’s future is liable to be misleading.

The approach taken by some of the big manufacturers also didn’t help: when the Vive and Oculus launched their supporting ecosystem of games and applications was comparatively weak. Only Sony really offered a substantive ecosystem for the Playstation – and even this was derided in some sections of the VR media for being “merely” VR ports of existing games.

Sansar a city street scene created by Paul Lapointe Credit: Linden Lab

Another aspect which potentially hasn’t helped VR to date is the “room-sized” versus the “seat” VR experience. As noted above, existing games ported to a VR environment is looked down on by many in the VR media, who have preferred to focus on all the juicy tech of room sensors, motion trackers, and associated gizmos which offer a “truly immersive” experience.

But room-sized VR predicates itself on people having the room to indulge themselves and / or the willingness to spend time setting-up / taking down their wonderful gizmos. And what does all this emphasis on freedom of movement say to those who aren’t gifted with good mobility?  So is room-sized VR really the be-all of VR at home?

Those Alexander speaks to tend to think not, preferring to point to VR needing both. This is something which is picked-up in the second part of the series, Where do we go?, which also brings Sansar into the frame of the discussion.

Unity CEO John Riccitiello

As with the first part of the series, Alexander opens Part 2 with another level-headed analysis of to how fast VR is liable to develop. Unity CEO John Riccitiello, for example, doesn’t see VR really starting to take off until 2018 or 2019.

His view is echoed  by Tim Sweeney, CEO at Epic. He again cites the need for improved hardware, with more favourable price-points  as being essential for the high-end VR market, something he doesn’t see forthcoming for a “couple of generations”. This puts his view in roughly the same 2-3 year time frame as Brendan Iribe at Oculus VR, who has indicated it’ll be around that long before his company will have their next generation hardware on the market.

The core of this part is an examination of two emerging aspects of VR: the “out-of-home” experience and “social VR”.

The former is the idea that rather than perhaps having dedicated space at home in which to experience VR, people will instead head off to the local “VR arcade” or “VR theatre” to enjoy a fully immersive experience of some description. This might sound fanciful, but The Void, a New York and London-based out-of-home VR experience has seen OptiTrack, the company behind much of tech used in the game, see an “explosion” of sales in the technology.

The Void is spectacular,” Alexander quotes says Unity’s Riccitiello. “I think we’re going to see hundreds of these dedicated locations for entertainment. Imagine a room four times this big. Here is the bar and there are six different experiences that are available around the room. I would definitely go. Imagine, In 1000 square feet you could have DisneyLand. All of it.”

When you think of the potential for not only immersive, group gaming environments, but things like group training and simulation, out-of-home centres could become a practical part of the entertainment and business landscapes, offering low-cost access to a wide range of VR environments and experiences for the public and clients.

For “social VR”, the emphasis very much turns to Sansar. While he doesn’t directly praise the platform, it’s fairly clear he sees Sansar, with its potential to truly democratise how people can build their own VR spaces as a potential cornerstone of the home VR market. If it can truly replicate SL’s “secret sauce” in giving users genuine creative freedom in bringing truly tailored and personal VR experiences into their homes, free from the filtering of how professional VR developers and studios think people want to have them, then the future potential with the platform could be enormous.

Through both parts of his examination of VR, Alexander offers much to read and consider, particularly for those who have extended doubts about VR and how it might fit the broader scheme of things (at least in terms of entertainment). As such VR Check-In Part 1 and Part 2 are well worth a side-by-side read.

VR and MR catch-up: March 2017

The Oculus CR-1 - US $100 drop in price, and US $100 reduction for Touch controllers
The Oculus CR-1 – US $100 drop in price, and US $100 reduction for Touch controllers

Here’s another rapid-fire update on the worlds of VR and MR as I’ve been able to track a few things.

Oculus Rift

Oculus VR has dropped the price of the Oculus Rift headset and the Oculus Touch controller by $100 each. At launch, the headset cost Us $599 and the controllers, released later and a crucial part of the system, were priced at US $199. The new pricing brings the price for both down to the US $598 mark – just US $100 more than the Playstation VR bundle, and makes the Rift headset much cheaper than its main rival – the HTC Vive.

“Cheaper” is of course a relative term. Despite work to allow the Rift operate with lower-specifications systems (see my last round-up), to get the fullest out of the system you still need a heft PC with a hefty price.

There is still no news on when the untethered version of the Rift, with the project name Santa Cruz, will be ready for consumers. The only major update is that when it does appear, it will be marketed via Oculus VR’s “lower end” mobile division.

Speaking at the announcement of the price drop, Oculus VR’s former CEO (now head of Desktop VR), Brendan Iribe, indicated that the company is focusing on the next generation of VR systems, which he defines as being “a very big leap from where we are today”. However, consumers are unlikely to see anything on this front for at least another two years.

On February 1st, the ZeniNax case against Facebook / Oculus drew to a close, with the plaintiff being awarded US $500 million in damages over alleged code theft. While Facebook is seeking to have the verdict set aside, on February 24th, ZeniMax filed an injunction seeking to block Oculus VR from using the disputed code in its products. The news came via several outlets at the time, including Ars Technica, which pointed out that the injunction probably won’t succeed, but that if it does, it could be massively disruptive to both Oculus and Samsung, as the code is also used in the Gear VR.

HTC Vive 2?

Rumours are circulating that HTC are working on the “Vive 2”, an improved version of their headset.  Details have been sketchy and a little confused; one early report from November 2016 suggested the “Vive 2” would be a wireless / WiFi system, but given this came out shortly before HTC and Vive X Accelerator company TPCAST announced a “tether-less” WiFi kit for the existing Vive, (see me last round-up, linked to above) that report many have been incorrect.

However, other sources have indicated that “Vive 2” is in development, but has not release date. It is also said to have the internal code-name of “Oasis”. Has someone at HTC been reading Ready Player One?

In the meantime, HTC aren’t cutting the Vive’s price – but they are offering a new finance plan to help purchase it. They’ve also announced two new accessories: a Deluxe Audio Strap and the Tracker. Both are priced at a “mere” US $99. The Deluxe Audio Strap is in fact a rigid, Oculus-style head mount for the headset, complete with headphones.

Vive's new rigid head mount for the display, complete with audio headphones. Your for US $99
Vive’s new rigid head mount for the display, complete with audio headphones. Yours for US $99

The Tracker, due to ship in Q2 2017, is essentially a sensor unit  which allows game and hardware developers to turn real-life props into virtual weapons / gaming pieces, from guns to swords, to bats and so on.  Once connected to a peripheral, it allows the Vive’s lighthouse sensors to detect and track it, enabling it to be visualised in-game.

Headset Sales

As has been widely reported, sales of VR headsets have been far slower than the early hype predicted. No surprises there in many respects. Currently, Sony’s Playstation VR system is the highest-selling – but that’s just about to hit the million units mark. Oculus Rift and Vive are some way behind, with 243,00 and 420,00 unit sales respectively at the end of 2016.

This plateauing of sales has led to some pundits almost writing-off VR. However, while it would seem likely VR will be a niche product when compared to the everyday potential of Augmented Reality / Mixed Reality (AR / MR), it’s worth remembering that consumer-centric VR is only at the first generation stage. It is hampered by cost and the need to be hooked into a high-specification PC. Over time, some of these aspects – especially cost – will come down, encouraging more widespread interest / adoption, especially in those markets outside of games where VR could have a real impact: education, training, simulation, design, architecture. So it is perhaps a little premature to be pointing at current sales figures and declaring VR a “fad” or similar.

Continue reading “VR and MR catch-up: March 2017”