Enter the Snapdragon: Qualcomm and “XR”

The ODG R9 AR headset. Credit: Osterhout Design Group

It’s no secret that when it comes to augmented reality (AR) and virtual reality (VR), I’m swayed more towards AR and “mixed reality” (MR) as potentially being the “thing” of the future. Not, as I’ve often said, that I don’t believe in VR – it will in time grow to fill various niches and requirements. Rather, I just feel that AR / MR have a much wider field of application when it comes to impacting our daily lives.

I mention this because earlier in June I read an interesting piece by Dean Takahashi, examining Qualcomm’s emerging role in what they like to call “XR” – or “eXtended Reality”, which they define as a fusion of VR, AR and MR.

Qualcomm spells out the hurdles to ‘extended reality’ glasses offers a transcript  from a chat Dean had with Tim Leland, Qualcomm’s vice president of product management, on the company’s goal and the challenges they see in bringing headset-style devices to the market.

Qualcomm, of course, is the company behind the veritable Snapdragon family of processors. In 2016, they announced their intention to make the Snapdragon 835 chipset the heart of a new range of self-contained VR and AR devices. To that end, they are about to start shipping the Snapdragon 835 VR HMD to OEMs wishing to produce Android-powered VR headsets using the chipset and Google Daydream.

More particularly – from my perspective – Qualcomm has already partnered with Osterhout Design Group (ODG), to develop a range of Snapdragon-powered AR headsets. I first became aware of the first of these units, the R7, in mid-2016. Intended to be a heads-up AR system for enterprise solutions (selling at US $2,750), it has gained a degree of traction in a number of fields – hazardous environments (oil exploration and production, chemical production and pharmaceuticals, healthcare and surgery), and has been involved in tests helping the visually impaired.

In 2017, ODG are due to release two more units – the “prosumer” R8 (around US $1,500) and the “consumer R9 (at “sub-$1,000”). Again, these are Snapdragon 835 based, and will be fully self-contained units with Android and their operating system.

It is the self-contained aspect of such headsets which Qualcomm sees as being one of the keys to the future success of “XR”.

“Any type of cable is just a non-starter,” Leyland notes. “Fans will not exist. We think there might be a niche market for glasses that maybe stream to a PC, but that’s a small part of it. The big part is everything self-contained in a mobile device. All the visual processing systems are very close to the inertial sampling systems, so everything is very fast.”

Qualcomm see “XR” systems potentially becoming a mainstay of our daily lives, fusing VR and AR into a single headset unit which can meet a variety of needs at any given time, and which can also be used as the basis for specific use-cases.

A conceptual “first responder” XR headset for fire fighters. Credit: Qualcomm

In this Qualcomm see XR units being both general purpose and specific to market sectors. The company is already looking at a concept for a “first responder” headset for fire fighters. Containing night vision capabilities and thermal imaging sensors, the headset could allow fire fighters overlay their field-of-view with floor plans of the building they are in, helping them find their way through the smoke, while the thermal sensors warn of potential hotspots and possible behind-the-door risks of backdraft – and could even guide them to people trapped in a burning location.

A simplified rendering of the kind of information the “first responder” headset might provide. Credit: Qualcomm

For more general use, Qualcomm are looking at headsets which integrate much of what they’ve developed with the likes of ODG – multiple cameras, integral motion tracking, the ability to track eye and hand movements, etc., but in a very lightweight, unobtrusive form-factor with a low price point which makes them an attractive proposition.

Not that this is going to happen overnight. A refreshing aspect of Qualcomm’s view is that they are looking at a development / adoption curve measured in at least a decade. As Leyland notes, the ability to have AR and VR heads headsets exists today, but there are hurdles to be overcome before they are as ubiquitous as the mobile phone for many of the tasks we perform today.

Some of these hurdles are being independently addressed – 5G, for example, is expected to be of huge benefit to those uses which require a lot of rendering and so are latency intolerant. Others are going to take time to progress and solve:  display requirements – the vergence and accommodation conflict, human field of view (190ox130o) etc; common illumination); motion and tracking for intuitive head, hand and eye movements; and power and thermal issues.

The technical hurdles “XR” needs to overcome. Credit: Qualcomm

Leyland doesn’t see any of these hurdles as being problematic – he just emphases that the time frame required to solve them is not going to be as compressed as some of the more bullish predictions about VR growth made in 2016 would have us believe. Instead, he points to 2020 as still being a year when numbers of shipped headset units of all types is still measured in the hundreds of millions, although he does see it growing from there.

IDC VR shipment numbers (in thousands), have been seen as a means to question the reality of the VR market

But will these systems ever reach the ubiquitousness of the smartphone? Right now, going on the shipments of VR headsets some are quick to pooh-pooh the entire mixed reality (or XR if Qualcomm prefer) ecosystem in favour of alternatives. On the surface, they would seem to be right – but on a longer-term look? I’m not so sure. Again, this is where the much-hyped smartphone analogy with VR is misleading – as Leyland points out in talking to Takahashi.

While it is true the first “genuine” smartphone as we know them today only appeared a decade ago, the fact remains that it was founded on some three decades of cellular phone development. right now, headset capabilities are roughly in the “1990s” phase of that overall curve – so there is a way to go. As such, while headsets that more closely resemble glasses / sunglasses may not necessarily become as all-pervasive as smartphones are today, there is little reason to doubt they could – if they have an intuitive ease of use – take over from handsets (and associated wearables) for a wide variety of tasks / uses.

Qualcomm isn’t alone in pursuing a convergent future of mobile VR / AR / MR capabilities. However, through Dean Takahashi’s article (and courtesy of Qualcomm’s Augmented World Expo presentation, it is good to see how level-headed is the approach being taken be tech companies to bot understand the technology , its potential and to look beyond the buzz phrases like “killer app” or order to make “XR” work.

Advertisements

HTC Vive and Microsoft HoloLens available for pre-order

The HTC Vive and Microsoft HoloLens: available to pre-order (sort-of in the case of the HoloLens)
The HTC Vive and Microsoft HoloLens: available to pre-order (sort-of in the case of the HoloLens) – credits; Vive via HTC, HoloLens via Microsoft

Monday, February 29th 2016 saw HTC / Valve open the doors for pre-ordering of the consumer version of the Vive VR headset, while Microsoft started accepting pre-orders for the Development Edition of the AR / mixed reality HoloLens system.

The first batches of each system are expected to start shipping around the same time as Oculus VR commences the first shipments of the Rift headset, which was made available for pre-order in January: the Rift is expected to start shipping on March 28th, world-wide, with the Vive starting on April 5th, also world-wide (although the latest update on the UK order page now states shipping will be in May 2016, possibly as a result of initial order received). The HoloLens will commence shipping on March 30th – but only to developers in the USA and Canada.

HTC initially announced the US consumer price for the Vive  – US $799 excluding sales and shipping –  on Sunday February 21st. This is some US $200 more than the Oculus Rift, but the prices does includes two wireless hand controllers; Rift buyers will have to purchase similar controllers separately, either from a third-party or through Oculus VR when their Touch system launches some time in Q2 2016. While no prices have been confirmed for the latter, many are taking Palmer Luckey’s comments that bundling Touch with the Rift would have “significantly” raised the price of the latter to mean that Touch is liable to cost between US $100 and US $200 – markedly closing the gap between the two systems.

The Vive pre-order kit comprises the headset unit, two wireless hand controllers, two room sensors and a pair of ear buds (the headset includes a jack socket for those wishing to use their own headsets / ear buds
The Vive pre-order kit comprises the headset unit, two wireless hand controllers, two room sensors and a pair of ear buds – although the headset includes a jack socket for those wishing to use their own headsets / ear buds (credit: HTC)

On February 28th, 2016, HTC further announced the Vive’s international pre-order pricing. This see the Vive pitched at £689 (around US $960) in the UK and €899 (US $977) in Europe, both inclusive of VAT but exclusive of shipping costs (£57.60 for UK customers). Customers in Canada can expect to pay CAD $1149 + tax and shipping.

The Vive package includes the headset, which has a similar technical specification to the Oculus Rift (but with a 9:5 aspect ratio rather than 16:9, the former being said to result in a more natural and convincing “feel” to images on the headset’s screens), the two wireless controllers,  a pair of Vive base station sensors, a Vive Link Box, and a pair of Vive ear buds. For a “limited period” pre-order units will additionally ship with two free VR games: Job Simulator: The 2050 Archives and Fantastic Contraption and will also include Google’s Tilt Brush VR painting system.

As an added sweetener for developers, and as reported by Tech News World, Unity Technologies has announced their game platform will have native support for the HTC Vive and Steam VR, while Valve have introduced an advanced rendering plug-in developed for Unity. There is also a Vive Developer’s portal, which includes support for Unreal Engine.

The computer hardware specifications for the Vive also pretty much resemble those of for the Oculus Rift, and like Oculus VR, Steam are offering an application that potential purchasers can download to test whether their PC is “VR ready”, while HTC offer a page of recommended PC hardware suppliers who can provide “Vive optimised systems” to US customers.

The Vive Pre consumer edition now available for pre-order
The Vive Pre consumer edition now available for pre-order (credit: HTC)

As I reported in January, the headset includes two interesting additions. The first is the front mounted “pass through” camera, which allows the user to see an overlay of the room around them projected into their virtual view. This fades in if they approach a physical object (e.g. a wall or desk, etc.), or can be manually triggered via the hand controllers, and allows for collision avoidance when using the headset with the room sensors to move around within a VR environment. The second is “Mura correction” (“mura” being a Japanese term meaning “unevenness” or “lack of uniformity”), which removes the inconsistent brightness levels between one pixel and the next on earlier Vive headsets, presenting a far more uniform and cleaner image.

Further information can be obtained from the HTC Vive pre-order website and via Valve’s Steam website.

Continue reading “HTC Vive and Microsoft HoloLens available for pre-order”

Ebbe and Eric discuss the future of VR with Bloomberg

Emily Chang from Bloomberg Business discusses the future of VR with Ebbe Altberg and AltspaceVR CEO Eric Romo
Emily Chang from Bloomberg Business discusses the future of VR with Ebbe Altberg and AltspaceVR CEO Eric Romo (via Bloomberg Business)

On Monday, December 7th, Linden Lab CEO Ebbe Altberg appeared alongside AltspaceVR’s CEO and founder, Eric Romo  on Bloomberg Business with Emily Chang, to discuss How to Find Realistic Timeline for Virtual Reality. In the interview, which lasts just under 5 minutes, the three discussed the potential of VR including.

The foundation for the interview is a report by TrendForce which proclaims the VR market will be worth around US $70 billion by 2020, with some US $20 billion coming from hardware purchases and US $50 billion from software and applications. It’s the latest in a bullish series of predictions on the future of the technology, many of which have gone unchallenged – and even then, TrendForce believe their prediction is an “understatement”. But how likely is it?

The Trend Force prediction for VR growth (via Bloomberg Business)
The Trend Force prediction for VR growth (via Bloomberg Business)

US $70 billion represents a tenfold increase in market worth for an industry slated to generate around US $6.7 billion in 2016. However you look at it, that’s a pretty steep growth curve. Both Ebbe and Eric see it as “reasonable”, with the latter citing the idea that a lot of companies which might not be considered as “VR companies” seeing a value proposition in the technology and leveraging it within their business model. In particular, he refers to the expected upsurge in VR as a paradigm shift comparable to that witnessed with the smartphone revolution.

Others are more cautious, as is the case with Oculus VR CEO Brendan Iribe, who is shown commenting:

We definitely believe the mass market … there’s going to be a lot of adopters, early adopters, of VR. but if you’re looking at the kind of smartphone scale, you know, billions of users out there, that’s going to take a long time.

While not nay-saying the potential of VR, other analysts view the TrendForce report as being unhelpful. When approached by Tweak Town, for example, Moor Insights & Strategy’s VR Analyst Anshel Sag, had this to say:

$70 billion by 2020 is more than extremely ambitious, $70 billion assumes that VR is a mature and mainstream market. There is no way that VR will explode into such a mature market within effectively 4 years. While I am extremely optimistic about the future of VR, such projections do nothing but hurt the future of VR by setting unrealistic expectations. There are no players in any part of the market that could turn this industry into a $70 billion industry in 4 years.

During the Bloomberg discussion, there is an acceptance that VR needs to overcome certain technical hurdles to gain more of a mass-market appeal. Certainly, these issues – cost, reliance on high-end supporting technology, etc. – are real, and doubtless will be overcome. But they aren’t the single issue facing VR in terms of its adoption.

Like it or not, VR is actually an isolating experience. Sure, you can in theory see anything, go anywhere, etc., while using it. But you do so at the expense of pretty much cutting you off from the rest of the world around you. It curtails your ability to properly interact with the things around you, to multi-task, etc. For many people and situations, even those seen as potential VR use-cases, that could curb the appeal.

There’s something else as well to be considered when discussing VR and its potential; what might be called the elephant in the room: augmented reality.

While AR is off to a slower start that VR, it is fair to say that it has the potential to reach into many of those markets and use-cases as seen to be ideal for VR, and offer a more attractive option in doing so. Initial AR systems are far more self-contained and portable; those on the horizon promise a wealth of capabilities (up to and including VR). More to the point, they do not isolate users from the world around them, something which could make AR far more practical and appealing for everyday use in the house, at work, on the street, etc.

By the time VR is really in a position to offer low-cost, lightweight systems freed from requiring high-end computing power, it could be facing stiff competition from AR for many of the markets seen as "ideal" for its use
By the time VR is really in a position to offer low-cost, lightweight systems freed from requiring high-end computing power, it could be facing stiff competition from AR for many of the markets seen as “ideal” for its use (image via CastAR)

So, it could be said that AR appears to be a far more natural proposition for widespread adoption and use, becoming a far more natural evolution from (and with) mobile and smartphone technologies. Hence why some put AR’s market worth as being in excess for US $100 billion by 2020.

Which is not to say that VR doesn’t have a place in the future. There are very niche and compelling cases where it will gain momentum. But whether it will ever reach the level of adoption comparable to the smartphone, as is so often cited, is questionable. There is no reason why, that for many of those potentially uses of VR outside of entertainment and gaming, AR might not offer a far better value proposition for take-up when compared to VR, leading to the latter being subsumed by it well before it has the opportunity to reach the scale of growth predicted for it.

You can catch the Bloomberg video by flowing the link towards the top of this piece, or you can catch the audio below.

Magic Leap reveals a little more about their AR system

Romy Abovitz talks Magic Leap at WSJD Live (image courtesy of engagdet)
Romy Abovitz talks Magic Leap at WSJD Live (image courtesy of Engagdet)

Magic Leap, the augmented reality company established by enigmatic entrepreneur Rony Abovitz, and which gained over US $500 million from the likes of Google in 2014 (see my article from October 2014),  revealed a little more about its product during a WSJD Live interview recently.

As reported by Nicole Lee for Engadget, Abovitz and Chief Content Officer Rio Caraeff talked in general terms about the system which has caused no small amount of excitement among those who have actually seen it in action. People such as Google’s CEO Sundar Pichai, who sits on the Magic Leap board and was one of the driving forces behind Google’s lion’s share of the half-billion funding mentioned above, and who shares a place on the board with Qualcomm’s executive chairman, Paul Jacobs, another investor from that round of funding.

The technology has also wowed leading lights from the entertainment industry such as Weta Workshop’s co-founder Richard Taylor (one of the first to invest in Magic Leap), and Thomas Tull, CEO of Legendary Entertainment (aka Legendary Pictures). For those outside, however, Magic Leap has perhaps come over as a riddle bordering on smoke and mirrors.

Magic Lap's website splash screen (no pun intended)
The Magic Leap website splash screen (no pun intended)

Talking at the WSJD event, Abovitz indicated Magic Leap won’t be tied to a particular platform or OS, but will be entirely self-contained and with a dedicated OS. He also indicates that in difference to those dismissing AR on the grounds that “no-one likes to wear glasses for long”, the form factor for the unit will be something people won’t fight shy of wearing. He also states it will allow them to maintain a normal relationship with others when in use – a little dig, no doubt, at the first generation of cumbersome and isolating VR headsets about to hit the market.

Alongside of the interview, Magic Leap also released a new video which they state is filmed entirely through Magic Leap technology and with no special effects or compositing.

Quite how the system works, however, is still a mystery. No headsets are shown in any of the company’s promotional material, and much is made of the use of a “Dynamic Digitized Lightfield Signal” (which we can call “Digital Lightfield™”). In speaking at WSJD, Abovitz and  Caraeff both skirted specifics, with the former only saying, “We treat human biology as our centre point; everyone already has a head-mounted display. It’s your head!”

This suggests the company is perhaps pursuing direct retinal projection, possibly in some form of headset unit that is less intrusive than the kind of units suggestive in the patent filings the company has made in the past. And if this sounds like science-fiction, remember Abovitz made his money developing medical technology, and the company has apparently devoted considerable effort into researching the relationship between the photonic light field and the way the brain functions.

Patent filed for Magic Leap by , draws extensively on one of earlier patents for augmented reality headsets styled in both "regular glasses" and "wrap-around" form factors (US 20120162549 A1)
Patent WO 2014043196 A1, filed for Magic Leap by Chunyu Gao for augmented reality headsets styled in both “regular glasses” and “wrap-around” form factors, suggesting some form of headset will be a necessary part of the system – click for full size

So what is the purpose of all this? Caraeff indicated the ultimate am is for Magic Leap to provide broad-based platform for visual computing. “Anything that you can do on your smartphone, on your computer; you’ll be able to do on Magic Leap,” he said, then added, “Where the world is your screen.”

“We believe the future of computing should be natural,” Abovitz stated. “With Magic Leap, your brain doesn’t distinguish what’s real and what’s Magic Leap, because as far as your brain’s concerned, it is real.”

I admit to being far more persuaded that AR will generate a greater mass market presence than VR. Despite the negative memes about people not liking glasses and Google’s misplaced Glass product, AR would appear to be far more inclusive in its use than VR, and have the same potential reach into many of the markets being hailed as VR’s territory: business, medical, education, healthcare and entertainment.

Whether Magic Leap will actually pave the way in this regard as units start to roll off the company’s new production line in Florida at some point in the future, is open to debate. I do, however, admit to being more intrigued by the potential of AR systems like it and CastAR than I am with the first generation of VR headsets we’re about to see.

Additional material on the WSJD Live event via The Verge.