Category Archives: Other Worlds

High Fidelity: September update and things to come

HF-logoThe September newsletter from High Fidelity appeared at the end of that month, with Chris Collins highlighting some of the work that has been going on of late, providing an update on particle effects, procedural textures and – most interestingly – avatar kinematics and in-world object manipulation using an avatar’s hands and via suitable controllers.

Procedural textures allow for complex, algorithm based textures to be created using tools such as ShaderToy and used directly within High Fidelity. Brad Davis has created a video tutorial on procedural entities which Chris references in the newsletter, the write-up also follows a short video released on the High Fidelity  you Tube channel which briefly demonstrates procedural textures in HiFi.

However, it is the object manipulation that’s likely to get the most attention, together with avatar kinematics and attempts to imply a force when moving an object.

In terms of avatar kinematics, Chris notes:

In 2016, when the consumer versions of the HMD’s are released, you are also going to be using a hand controller. It is therefore important that we can make your avatar body simulate correct movement with the hand data that we receive back from the controllers.

The results are shown in the newsletter in the form of  some animated GIFs. In the first, Chris’ avatar is shown responding to a Hydra controller for hand movements and echoing his jaw movements. The second demonstrates object manipulation, with Chris’ avatar using its hand to pick up a block from an in-world game, echoing Chris’ motions using a hand-held controller.

Manipulating in-world objects in High Fidelity via an avatar's hands and a set of controllers (image: High Fidelity)

Manipulating in-world objects in High Fidelity via an avatar’s hands and a set of controllers (image: High Fidelity)

The animation in picking up the block may not be entire accurate at this point in time – the block seems to travel through the avatar’s thumb as the wrist is rotated – but that isn’t what matters. The level of manipulation is impressive, and it’ll be interesting to see if this might be matched with things like feedback through a haptic style device, so that users can really get a sense of manipulating objects.

The object manipulation element, together with attempts to imply a force when moving objects in-world which make up a core part of the video accompanying the newsletter (and which is embedded below). Again, this really is worth watching, as the results are both impressive, and illustrate some of the problems High Fidelity are trying to solve in order to give virtual spaces greater fidelity.

Coupling object manipulation with implied force opens up a range of opportunities for things like in-world games, physical activities, puzzles, and so on. There’s also potential for learning and teaching as well, so it’ll be interesting to see how this aspect of the work develops.

The newsletter also promises that we’ll be seeing some further VR demo videos from High Fidelity in October, so keep an eye out for those as well.

High Fidelity: into the solar system and STEM grant recipients

HF-logoI’m rather into space and astronomy – that much should be obvious from my Space Sunday reports, and coverage of mission like the Curiosity rover, astronomical events like the transit of Venus and so on.

So when High Fidelity posted news on the 2015 summer intern project, and the words “solar system” featured in it, my attention was grabbed. The post opens:

Hello! I’m Bridget, and I’ve been interning at High Fidelity this summer, working to build some JavaScript content in HF. As a math and computer science major, I had the opportunity to hone my programming skill set, learning from Hifi’s superb team of software engineers and design-minded innovators.

So here’s the culmination of my work this summer: a virtual orbital physics simulation that provides an immersive, interactive look at our solar system.

Bridget's solar system model correctly simulates the movement of planetary bodies around a stellar object , utilsing both Newton's and Kepler's laws, thus producing a dynamic teaching model for orbital mechanics and gravity

Bridget’s solar system model correctly simulates the movement of planetary bodies around a stellar object , utilising both Newton’s and Kepler’s laws, thus producing a dynamic teaching model for orbital mechanics and gravity – with a potential application for teaching aspect of physical cosmology

The goal of Bridget’s project is to demonstrate what can be built using JavaScript (and some C++), with a particular emphasis on building educational content in High Fidelity, and by using the solar system, she has come up with a highly innovative approach to teaching orbital mechanics – and more besides.

Essentially, she has created a model of the solar system which uses “real” gravitational physics to simulate the motion of the planets around the Sun. The planets themselves occupy orbits scaled relative to Earth, and fixed reference values are used for the orbital period, large and small body masses, and gravity. Then, a little Newtonian physics is thrown into the mix, together with a sprinkling of Kepler’s Laws of planetary motion. Thus, the scripting ensures that the planets maintain a stable orbit, while updates correct mimic each planet’s orbital trajectory around the Sun.

This generates a model that is interesting enough in itself, if somewhat simplified in nature, as Bridget notes, whilst also pointing to its potential for further use:

While the simulation exploits a somewhat simplified model, namely neglecting the elliptical nature of the planets’ orbits, it can easily be modified to account for additional factors such as the n-body problem.

In other words, there is the potential here to both refine the model in terms of orbital mechanics and planetary motion as a part of the teaching / learning process, and perhaps even dip a toe into physical cosmology.

the simulation

the simulation includes a UI which allows users to perform a number of tasks, including playing a little game and being able to zoom into the planets.

Bridget also notes:

Another fun aspect of the project was implementing UI to create possibilities for exploration and experimentation within the simulation. A panel with icons lets you:

  • Pause the simulation and show labels above each planet revealing its name and current speed
  • Zoom in on each planet
  • Play a “Satellite Game” (think Lunar Lander, but with a satellite around the earth), where you attempt to fling a satellite into stable orbit
  • Adjust gravity and/or the “reference” period, and see what happens!

Bridget’s work marks the second time a summer intern has reported on working at High Fidelity during the summer hiatus. In 2014, Chris Collins chatted to the (then) 17-year-old Paloma Palmer, a high School student also honing her coding skills. She focused on coding voxels to respond directly to volume inputs over a microphone in real-time. You can see her discussion with Chris on the HiFi YouTube channel.

Staying with education, and following on from my coverage of High Fidelity’s STEM VR challenge, Ryan Kampf announced the first of the grant recipients on Friday, August 14th.

The VR challenge invited educators, be they individuals or groups, to take up the STEM VR Challenge, to submit proposals for educational content in High Fidelity which meets the criteria set-out in the Challenge website, namely that the content is:

  • HMD (e.g. Oculus Rift) featured
  • High school age appropriate
  • STEM focused
  • Social (can be experienced by >3 people together).

On offer were up to three grants of US $5,000 each for recipients to further develop their ideas.

In his  announcement Ryan indicated that two recipients for grants had been selected from submissions: the TCaRs VR Challenge and Planet Drop VR.

Both use game mechanics, with TCaRs (Teaching Coding – a Racing simulation) enabling users get to interact with and customise their racing cars using JavaScript, while Planet Drop places players into an alien planet environment which they must explore through “cooperative asymmetrical gaming”. Each has highly specialised information, based on their chosen STEM field and provided to them via a game HUD, and the aim is for them to work together, sharing the information they receive as quickly and effectively as possible to allow the team to solve challenges and advance through a story arc of increasingly impressive accomplishments.

Conceptual illustration of the "Mech Pods" the players in Planet Drop will use to explore their alien environment

Conceptual illustration of the “Mech Pods” the players in Planet Drop will use to explore their alien environment

Congratulations to Bridget on her summer intern project (the script is available for those wishing to use it), and to the STEM VR challenge recipients.

OpenSimulator: Justin Clark-Casey steps back

Maria Korolov on Hypergrid Business covers the news that Justin Clark-Casey is significantly scaling-back his involvement in OpenSimulator development.

For those deeply entrenched in Second Life, his name may well pass unnoticed. However, since 2007, Justin has been deeply involved in OpenSimulator, as both a core developers and as a founding member and first president of the Overte Foundation, a non-profit organisation that manages contribution agreements for the OpenSimulator project.

Just how big a role he has played can in part be seen through the 11,631 code commits he has personally made to the project over eight years  – that averages out to just under four commits every single day.

Justin announced his decision to step back from what has been a central role within the OpenSimulator in a blog post, where he emphasised that he’s doing so in part because he’s shifting career, although he makes it clear he is not leaving OpenSimulator entirely; it just won’t be a primary focus in his life in the foreseeable future:

OpenSimulator (and the Metaverse in general) has been an amazing journey but, as they say, we have grown apart. For whatever reason the area doesn’t fascinate me as it did. For better or for worse, that’s crucial for me to feel happy in my work.

I’m not disappearing completely but very likely for the immediate future my involvement will be at a low ebb (mainly answering mailing list questions and the occasional bug fix). My new field is quite a bit different (data warehousing for genetics and synthetic biology) but I will always have a soft spot for virtual worlds and the idea of the Metaverse.

Justin Clark-Casey's code commits to OpenSimulator amount to 11,631 over eight years, work that has involved him in laying many of the foundations for the project and in re-factoring much of the code-base in 2011/12 (source: Black Duck’s Open Hub open source project tracker, via Hypergrid Business)

Justin Clark-Casey’s code commits to OpenSimulator amount to 11,631 over eight years, work that has involved him in laying many of the foundations for the project and in re-factoring much of the code-base in 2011/12 (source: Black Duck Open Hub open source project tracker, via Hypergrid Business)

As well as his own code contributions, Clark-Casey has been noted for carrying out a significant portion of the work required integrate patches submitted by others, and has also taken on many of the organisational duties and activities which have perhaps been seen as somewhat onerous by other developers.

His popularity and import to the OpenSimulator community can be measures by the outpouring of personal thanks and testimonials which followed his own blog post and featured in Maria’s Hypergrid Business article.

According to Maria, Justin’s announcement has led to some concerns as to the future of the project. While there has never been a single de facto leader for the platform and its very diverse and global community, Clark-Casey has very much been the public face of the platform, hence some of the concerns raised.

However, as others central to the platform’s development have been quick to point out, this is not the first time a key figure has opted to set back from the platform. As it is, the team of core developers has changed over the years and remains strong. Similarly, OpenSimulator itself enjoys broad-based support and engagement from individuals, groups, education and academia and business. As such, there is little need to doubt its foreseeable future.

“Open source development has a high churn of people, for many reasons, and many times people who have been there for a long time simply decide to leave and do something else,” Crista Lopes, creator of the Hypergrid, told is quoted as saying in Hypergrid Business. “The good thing about open source projects is that, if people find them useful or interesting, the projects survive any one particular developer’s absence. That will happen with OpenSim too.”

I only had cause to talk to Justin twice over the years, and was certainly not in any way acquainted with him. However, as a very occasional OpenSimulator visitor (notably via Kitely, OSGid and InWorldz), I offer my own thanks to him for all of his contributions to the OpenSim community, and best wishes as he enters a new stage in his career.

Related Links


The Drax Files Radio Hour: giving it the HiFi!

radio-hourOne of the big use-cases is going to be kids maybe doing an extra, like instead of doing their homework in the normal way in the evening, they go on-line where they join a study group where they join a teacher..

So opens segment #75 of the with some thoughts from Philip Rosedale, co-founder of Second Life, and more particularly now the CEO of start-up virtual worlds company, High Fidelity.

At just over 89 minutes in length, this is a special show, exploring High Fidelity from the inside, so to speak, complete with conversations with Mr. Rosedale, Ryan Karpf (HiFi’s co-founder and ex-Linden), Chris Collins and Ozan Serim, while David Rowe (perhaps more familiarly known to SL users as Strachan Ofarrel creator of the Oculus Rift compatible CtrlAltStudio viewer), who has been working with the HiFi team, becoming a guest host for the segment.

Since its founding, High Fidelity has made remarkable strides in developing its next generation, open-source virtual world environment, both technically and financially. Since April 2013, the company has undergone three rounds of funding, attracting around US $16 million, most of which has come from True Ventures, Google Ventures and, most recently, Paul Allan’s Vulcan Capital (which also participated in the October 2014 US $542 million investment round for Magic Leap). In addition, HiFi has attracted a number of high-profile advisers, including VR veteran Tony Parisi and, most recently, professors Ken Perlin and Jeremy Bailenson.

As well as Philip Rosedale, Drax talks with Chris Collins (l), Ryan Kampf and Ozan Serim from high Fidelity

As well as Philip Rosedale, Drax talks with Chris Collins (l), Ryan Karpf and Ozan Serim from high Fidelity

The interviews themselves are quite wide-ranging. With Dave Rowe, (known in HiFi as CtrlAltDavid) the open-source nature of the platform is explored, from the ability to download and run your owner HiFi server (aka “Stack Manager“) and client (aka “Interface“), through to the concept of the worklist, which allows contributors to bid for work on offer and get paid based on results.In Dave’s case, this has led him to working on various aspects of the platform such as integrating Leap Motion capabilities to improving eye tracking within HiFi’s avatars, so they track the movements of other avatars, just as our own eyes track other people’s facial and other movements as they interact with us.

In terms of general looks, the avatars – which have in the past been critiqued for being “cartoony” (despite it is still very early days for HiFi) –  are still very much under development. In particular, Ozan Serim has been working to raise –  and no pun intended here – the overall fidelity of the avatars in terms of looks and capabilities. He’s well-placed to do so, being an ex-Pixar animator.

One of the problems here is that the more real in appearance and capabilities they get, the closer the avatars come to the Uncanny Valley, which has led HiFi and Ozan to look at a number of avatar styles, from those which are very human in appearance through to those that are more “cartoonish” in looks.

A 2014 video showing Ozan’s work in improving the rigging around a more “realistic” HiFi avatar to more actually reflect mouth forms and facial movement when singing. High Fidelity now use Faceshift for real-time facial expression capture, rigging and animation, using either 3D or standard webcams

In discussing the Uncanny Valley, and particularly people’s reactions to avatars that are somewhat less-than-real (and we can include SL avatars in this, given their inability to naturally reflect facial expressions), Ozan raises the interesting question of whether people who critique the look of such avatars actually want to have a “realistic” looking avatar, or whether it is more a case of people wanting an avatar look that is appealing to their aesthetics which they can they identify with.

This is and interesting train of thought, as it is certainly true that – limitations of the avatar skeleton aside – most of us in Second Life are probably more driven to develop our avatars to a point where they have a personal aesthetic appeal, rather than in wanted them to be specifically “more realistic”.

Currently, HiFi is leaning towards a somewhat stylised avatar as seen in Team Fortress 2, which is allowing them to develop a natural-looking avatar look that doesn’t come too close to the Uncanny Valley. They use Adobe Maximo as their avatar creation tool, which Ozan views as a capable workflow package, but which may have some creative limitations. However, as an open-source environment, HiFi does offer the potential for someone to script in “in-world” character modelling tools, or at least to offer upload capabilities for avatar model generated in tools such as Blender. Avatars can also, if wanted, by uploaded as a complete package with all required / defined animations, such as walks, etc, included.

Chris Collins has very much become the voice of High Fidelity on You Tube, producing a wide range of videos demonstrating features of the platform, together with short tutorial pieces. The video above is one of his, demonstrating how to code interactive 3D content, using the Planky game as an example

While Ozan and his team work on avatar animations and rigging using real-time capture, Ryan Karpf reveals that by default, an avatar’s facial expressions are driven via the audio more than by direct capture: the mouth movement, for example, comprises 3 positions based on the audio, while a rising of voice or tone can result in the avatar’s eyebrows rising and falling. Ryan also touches on the Uncanny Valley issue of people’s increasingly discomfiture the closer avatars become to looking “photo-realistic”.

In talking to Chris Collins, an ex-Linden Lab alumni who headed the former SL Enterprise division, who now wears a number of hats at HiFi, Drax discusses how HiFi deals with the ever-changing face of the emerging VR hardware market, where headsets, input, tracking, and so on, is in something of a state of flux. Chris points out that while open-source, HiFi does have a set of strict coding standards and licensing, and offer external libraries to help support third-party SDK integration.

One of the powerful elements of High Fidelity is the ability you to have full agency over your environment, if you so wish; using the Stack Manager, you can create your own server / world / space, and control who might access it.  The scripting tools similarly allow users to download and tweak elements – such as walking animations, a basic avatar appearance, etc., quickly and easily.

Continue reading