Talking castAR and High Fidelity

The Silicon Valley VR (SVVR) Meet-up at the end of March featured a series of presentations from people within the VR field, including those by Brian Bruning, VP of Business Development and Marketing at Technical Illusions (castAR) and Philip Rosedale of High Fidelity.

The full video of the presentations is provided below, and I’ve included notes on each of these two presentations in particular. When reading, please be aware that these are notes, and not a full transcript.

Brain Bruning – castAR

Brian Bruning’s presentation commences at the 0:05:48 mark.

Image courtesy of Technical Illusions
Image courtesy of Technical Illusions

I’ve covered the early work on castAR in the past, some of which is touched upon at various points in the presentation, so I don’t want to repeat things here. What is interesting is that the system’s development has been following a similar route to that of the Oculus Rift: Technical Illusions have been out attending technology shows, conferences, exhibitions, etc., to gain visibility for the product , they ran a successful Kickstarter campaign for castAR which raised $1,052,110 of a $400,000 target.

[07;10] castAR has three modes of operation:

  • Projected augmented reality (AR), which presents a 3D hologram image projected onto a retro-reflective surface in front of you. allowing you to interact with it via a “wand”
  • Augmented reality of a similar nature to that of Google Glass
  • Virtual reality of the kind seen with the Oculus Rift.
castAR projected AR gaming with the castAR wand (image via Technabob)

The emphasis is that the headset is natural, comfortable-looking (a pair of glasses) which has three product features built-in. As a result of the Kickstarter, the company has now grown to 10 people, and the technical specifications for the system have been decided:


  • Less than 100 grams in weight
  • Fits over most prescription glasses
  • Ultra flexible micro coax cable
  • Active shutters with 50% duty cycle

  • 1280 x 720 resolution per eye
  • 120hz refresh rate per eye 24 bits of color per pixel
  • 65 degree horizontal field of view 93% fill factor
Tracking System

  • 110 degree FOV
  • 120hz update rate
  • 8.3ms response time
  • 6 degrees of freedom
  • Absolute positioning Over 200 unique tracking points
  • 0.07mm accuracy at 1.5m
AR & VR Clip-On

  • 90 degree horizontal FOV
  • Very low distortion freeform optics
  • 5mm by 8mm eye box
  • Removable flip-up shutter for AR mode

[11:20] castAR has its roots within the gaming environment and has been developed with the games market in mind (again, as had pretty much been the case with Oculus Rift), although they had recognised the potential for wider applications – they just hadn’t anticipated that someone like Facebook would step into the VR / AR arena and potentially add impetus to the wider applications for VR / AR.

[11:45] One of the benefits seen with a combined approach to VR / AR is that there are situations in work, in education, in research / medical fields where a completely occluded view of the real world  – as required by head-mounted displays (HMDs) such as the Oculus Rift – are simply not appropriate (Mr. Bruning jokes that there are even some activities associated with gaming where a HMD is inappropriate – such as simply trying to eat a snack or take a drink without interrupting the game flow!). In these situations, the projected AR or the Google Glass-like” AR are seen as more beneficial, and hence the drive to address all three modes of operation.

[13:20] Technical Illusions believe that many of the challenges faced by AR and VR content creators are similar in nature – such as dealing with UI issues, both seeing UI elements and interacting with those UI elements, or dealing with physical objects which my be places within a VR / AR scene. As such, Technical Illusions are focused on educating content creators to the needs of immersive / augmented environments and are producing dev kits to assist content creators in developing suitable environments / games / activities which take such issues into account.

[14:57] Current planning is for Technical Illusions to have their dev kits and the Kickstarter sets shipped in summer 2014, and to have the consumer version ready to ship by the fourth quarter of 2014, and it is indicated that price-point for consumer kits (glasses, tracking components, retro-reflective surface and input wand) will be “sub $300”.

The castAR update is an interesting, fast-paced piece, primarily focused on the projected AR capabilities of the glasses. Little or nothing was said reading the ability of the system to be used as a VR system, and no disclosure was given on the VR clip-on system.

This is apparently a deliberate decision on the part of the company, in that they are allowing VR HMD focused companies promote the potential use of VR, While Technical Illusions focus on the potential of projected AR capabilities.  While an interesting approach to take, I can’t help but feel that (assuming the VR clip-on is at a “feature complete” status) promoting all capabilities in castAR  wouldn’t be better, as they help present the product as a more versatile tool.

Phlilip Rosedale – High Fidelity

HF-logoPhilip Rosedale’s presentation commences at the 0:42:30 mark.

[42:53] High Fidelity are just about to start running their closed alpha, and the SVVR was seen as an opportunity to show some of what was going on in that programme. This entailed the use of hand-held devices as well as the camera system, which were used to track his hand and arm movements and display his hands as a part of the projected avatar.

The use of such sensors – able to capture facial expressions, movements, etc., are seen by High Fidelity as key to the future use of VR-style (and AR style?) technologies, especially with regards to getting people more immersed in the VR environment more rapidly than may be the case at present.

Philip Rosedale and avatar at SVVR (still via the SVVR video by )
Philip Rosedale and avatar at SVVR (still via the SVVR video by Matthew Carrell)

[44:30] The emphasis at High Fidelity has been very much on developing a more natural, realistic approach to avatar interactions: capturing facial expressions, hand movements, etc, as a means of enhancing the environment as a means for communications. and person-to-person interactions, as demonstrated in an exchange between an avatar / avatar interaction between Rosedale and HF employee Emily, which also touches upon spatial sound being a very important element with the experience.

Whispering sweet nothings. note how Rosedale has turned his head, his avatar has turned its head in response, and Emily is leaning in (to his avatar on her screen) to whisper into his ear
Whispering sweet nothings. note how Rosedale has turned his head, his avatar has turned its head in response, and Emily is leaning in (to his avatar on her screen) to whisper into his ear (still via the SVVR video by Matthew Carrell)

[47:25] System latency  – much of a focus of HF demonstrations and discussions – is raised, with Philip pointing to the system having a latency of around 100 milliseconds (compared to 500 milliseconds for cellphones and 700-900 milliseconds for VoIP). While oft-referenced, how much of an impact this really makes. The suggestion is that exchanges are more “real-time” and interruption thus more appropriate as a result of HF’s lower latency. However, I’m not entirely convinced an extra 400 milliseconds actually impacts my ability to interrupt or respond to someone when I’m talking to them on the ‘phone!

[49:00] High Fidelity allows for a more natural means of building – using JAVA Script – with suitable sensors, allowing one to effectively “paint” voxels (easiest analogy: think prim) in-world with hand movements – and these are witnessed in the same spatial space by anyone else with you in that space (they can also simultaneously edit what is being created). The building system appears to be flexible and interactive, and apparently includes support for 3D modelling and construction. The hope within High Fidelity is that the system is so flexible and intuitive that a catalogue of objects and items will quickly be built-up during the alpha phase.

[53:50] High Fidelity is intended to operate on a distributed computing basis, with people allowing the service to harness their own system’s (including cellphones) computing power when not in use, so you go to bed and keep your computer running, and HF harnesses its power to help create part of the virtual environment. Think The SETI@home project’s use of people’s computers, and you pretty much get the idea.

A night club with avatar bots running as assignments across 40 different machines and threaded through the HF servers respond as resonators to the audio stream - and to Philip Rosedale's voice - demonstrate the distributed computing approach taken by HF
A night club with avatar bots running across 40 different systems and threaded through the HF servers respond as resonators to the audio stream – and to Philip Rosedale’s voice – demonstrate the distributed computing approach taken by HF (still via the SVVR video by Matthew Carrell)

[59:50] HF will likely make its money through the provisioning of additional service layers – such as the sale of unique names (avatar / corporate / location). They will also create a currency / crypto-currency  which will cover goods and services (as with the L$ and SL) and also recognise the contribution people are making by providing the distributed computing needed to generate the world(s) within the HF network (so you earn credits by letting HF use your system). As a part of this, the company will supply the online marketplace by which goods and services can be sold to users of these interconnected worlds and environments.

Emily’s avatar (image courtesy of High Fidelity)

During the Q&A session (starting at 1:03:00 into the video), it’s interesting to note that while Rosedale mentions that High Fidelity works with Oculus Rift, he doesn’t view VR HMDs as being potentially as much of a game changer for VWs as some might expect. He places the emphasis far more on sensor systems (as they emerge) which allow users more freedom to express physical movement (hand, arm, etc.) in the digital environment as doing more to unlock the potential of virtual environments. That he feels this way shouldn’t be surprising; HF have already put considerable effort into facial expression mapping, etc., (up to and including addressing issues of the uncanny valley), work which is pretty much undone when someone slips a lump of inert plastic over the upper half their face.

So where does all this leave Second Life? Some have seen HF as being the logical successor to SL and that as such, there will be some kind of logical “upgrade” path between the two. I’ve always seriously doubted this, and Rosedale pretty much steered any from it in answering a question. Yes, LL have invested in HF, but this appears in no way to be a major investment (True Ventures and Google Ventures seem to be the major players). As such, it seems likely that while some of the applications HF are developing may be adopted by LL for Sl, any “upgrade” path would probably be limited to whatever you can export from SL (avatar shape files, your own creations) and then upload to HF (possibly after passing items through s suitable external modelling programme).

The matter of identity is touched upon – again an interesting topic given the FB acquisition of Oculus VR. Here Rosedale’s view would appear to be sharply divided from that of Facebook, as he sees the right to manage our identity and what we reveal of it to others must remain ours to determine and drive. For something like HF, this is a very complex subject given how they see the environment working; rather than a single VW, people are moving between worlds, and as such need to be able to carry with them more than just a name, but their physical identity in terms of looks and clothing – with the requisite surety that their identity is protected.

Overall, the HF presentation is interesting in that it puts meat on the bones of things a lot more. Exactly what can be achieved remains to be seen. I confess to still finding myself questioning some of the assumptions being made about VR / VWs, and am still less than convinced by some of the arguments being put forward as to how the brave new world will evolve. However, this article is long enough as it is without further meandering thoughts from me.

One thought on “Talking castAR and High Fidelity

  1. All this stuff is fantastic- really! But i think it’s putting cart before horse. Surely it’s lot more prudent to “just” get a usable HMD out first, and then add all the super-duper interactivity etc….!?


Comments are closed.