Why we’ll take VR and virtual worlds to other planets

The first touchdown: human missions to the surface of Mars have long been dreamed about and planned for. Sometime in the next 30 years or less, they’ll become a reality. And VR, AR and virtual worlds are likely to play a role (image: SpaceX)

Sometime in the next thirty years, it is likely that humans will set foot on the surface of Mars. The mission that takes them there might be an international government-sponsored mission, or it might be the result of private endeavour. However it comes about, it will be the culmination of decades of planning, hopes and dreams stretching back beyond the birth of the space age.

There is much that a crew on such a mission will be taking with them in terms of hardware, equipment and technology. And it is very likely that when looking down the list of technologies they’ll take with them, one will be able to find virtual reality, virtual worlds and augmented reality – an in a variety of roles and uses.

Take the crew’s psychological health and well-being for example. A round-trip mission to Mars will take between two and 2.5 years to complete, depending upon the “class” of mission undertaken.

The two classes of Mars mission: opposition (l), which are launched when Earth and Mars are on the same side of the Sun, and conjunction class (r) are launched when the Earth and Mars are on opposite sides of the Sun both amount to a mission duration of 2- 2.5 years
The two classes of Mars mission: opposition (l), which are launched when Earth and Mars are on the same side of the Sun, and conjunction class (r) are launched when the Earth and Mars are on opposite sides of the Sun both amount to a mission duration of 2 – 2.5 years

Throughout that entire time, they’ll be completely isolated from everything we take for granted here on Earth – the freedom to wander outdoors, the sight of a blue sky, green hills, rivers, the sea, cities, lakes, people; they’ll be confined to enclosed spaces which really don’t offer too much in the way of privacy. They’ll even be confined to meals from a menu set months in advance, with no real option to give into a whim for a particular delicacy if it isn’t on their vessel.

For the majority of the mission time, the only people they’ll be able to directly converse with are their fellow crew members – with a minimum round-trip time delay in communications between Earth and Mars of 8 minutes (and potentially as much as 40 minutes through parts of the mission), having real-time conversations with loved ones on Earth simply isn’t going to be possible; they’ll have to rely on pre-recorded messages and video and e-mail.

In these circumstances, stresses are bound to develop, both for the individual members of the crew and, potentially, between team members, no matter how carefully selected for compatibility ahead of the mission or how well-trained. One way of potentially dealing with them is through the use of VR and virtual environments, as NASA and other organisations have been investigating for much of the last decade.

It’s not hard to imagine, for example, a crew going to Mars with a library of pre-filmed environments and events  which they can then explore and enjoy individually or together through the use of personal headsets – or for such a library to be updated with new items beamed via  something like OPALS to their craft. Such environments and activities could provide psychological relief from the confines of the space vehicle.

In June 2014, NASA’s OPALS system beamed the high-definition, 36-second movie “Hello, World” from the International Space Station (travelling at 28,000 kilometres an hour (17,500 mph) to a receiver on Earth in just 3.5 seconds (compared to the 10-12 minutes radio communications would have required. Systems like OPAL offer the key to providing very high bandwidth communications capabilities between Earth and Mars, allowing much more data to be passed back and forth (image: NASA)

Similarly, high fidelity virtual world environments which support direct interaction, such as through haptic feedback mechanisms, might provide the means by which crew members can “remove” themselves from the confines of their vehicle and enjoy a variety of activities, including something we take for granted in VWs today – the ability to create and build.

ANSIBLE (A Network of Social Interactions for Bilateral Life Enhancement) was an initial attempt by NASA, working with SIFT and All These Worlds, to explore how virtual worlds might be leveraged to provide astronauts with environments which could be shared or used individually, and which might offer a range of AI interactions as well.

A screen capture of the main ANSIBLE environment. While openSim probably won't be the VW of choice for a Mission to Mars, the ANSIBLE environment has been used as a means of assessing virtual world environments could each the psychological pressures face by a a confined crew on a long duration space mission
A screen capture of the main ANSIBLE environment. While OpenSim likely won’t be the VW of choice for a Mission to Mars, the ANSIBLE environment is perhaps the first step towards assessing how virtual world environments could ease the psychological pressures face by a confined crew on a long duration space mission (image: SIFT / All These Worlds)

An intriguing element with ANSIBLE was the exploration of the idea that virtual world environments could be asynchronously “shared” between crew members and their friends and family on Earth, allowing them to engage in shared content creation activities, for example, through the swapping back and forth of OAR files, the ability to engage in “shared” immersive games and so on. ANSIBLE researchers even suggested that used in this way, a personal virtual world space could enable an astronaut and their family “share” special occasions more personally than could be done via e-mail, radio or video.

Commenting on the used of immersive environments and haptic technologies in Moving to Mars: There and Back Again (Journal of Cosmology, 2010, Vol 12), Sheryl L. Bishop, Ph.D, noted, “Telepresence and full fidelity audio/video/3-D communication replay capability will provide for more effective psychological support and interaction for crew members and to families and friends back on Earth.”

In terms of crew welfare, virtual reality has another potential use: assisting in matters of fitness. Most current mission scenarios involve the crew travelling to and / or from Mars in a “weightless” environment. Such an environment can be detrimental to many aspects of human physiology – muscles, bones, heart, lungs, etc. It is therefore essential long exposure to weightlessness is countered by routine exercise of up to two hours every day.

Exercise is an essential part of life in micro-gravity, where muscles can easily atrophy, bones suffer calcium loss, the cardiovascular system weaken, etc., away from the pull of Earths gravity. VR could help make such exercise more interesting and help space crews “escape” to more Earth-like environments (image: NASA)

In the confines of a space vehicle, the opportunities for exercise tend to be limited and potentially boring. How much more pleasant it might be for an astronaut who, after lugubriously strapping themselves into a treadmill harness and making all the required tension adjustments ready for 30 or so minutes of going nowhere while staring at a bulkhead, could slip on a VR headset, and go for a run through a woodland park or along a beach, the sounds of nature or the waves in their ears?

VR, AR and immersive environments have other applications within a mission as well.

NASA VR training system provides the means for astronauts to rehearse activities and gain familiarity with techniques. Such systems could be used be crews en route to Mars
NASA already uses VR to help train astronauts. A VR system carried aboard a vehicle en route to Mars could be used by the crew to stay familiar with techniques and activities they will – or might – have to carry out on Mars, or rehearse an activity (e.g. a repair to a surface vehicle) ahead of the actual EVA (NASA JSC)

NASA already employs virtual reality and haptic / kinetic systems to train crews heading for the International Space Station. not only would such techniques be used to train crews selected for a Mars mission, there is no reason why portable versions of these systems couldn’t be carried aboard their vehicle, offering additional opportunities for training and for “refresher” courses in various activities.

Further, data gathered via orbital and ground reconnaissance could be combined to create 3D representations of the landing site, allowing the crew to familiarise themselves with it at ground level, practice “driving” their rovers, etc. Equally, pre-defined immersive simulations could be used by the crew to rehearse critical parts of the mission, such as atmospheric entry, descent and landing at either end of the mission.

Another area of VR use is in what NASA calls “immersive telepresence”. This allows human operators to take direct control of  rover vehicles used in support of the mission, giving them added flexibility of use. While the rovers might operate largely autonomously, carrying out remote survey work or sample-gathering, should one find something of potential interest, a crew member could take command, using VR to “see” what the rover sees, and teleoperation to manipulate its robot arms.

NASA's use of telepresence with robotic vehicles:
NASA’s use of telepresence with robotic vehicles: ignore the human look of the robot rover, what matters here is the way the operator can use telepresence to control the rover, the headset presenting him with the view the robot has, the sensors attached to his hands allowing him to manipulate the vehicle’s arms and receive feedback on things such as the amount of grip and force being applied to the items being held, while the foot pedals help with  manoeuvring the vehicle (NASA JSC)

Augmented reality is also likely to play a variety of roles in long duration space missions. For example, it is likely to be incorporated, Google Glass-like, into space suits, providing the wearer with access to a wealth of additional data which can be called upon according to situation.

Another use for augmented reality already under development is again related to crew healthcare. A crewed mission to Mars may experience any number of medical problems arising from the likes of accidents or unforeseen health issues, some of which will need immediate medical intervention. While crews will undoubtedly include a surgeon-internist-astronaut, it is unlikely that they will be trained to handle every medical  procedure they may have to perform. Nor, due the communications delays already noted, are they going to be able to rely on direct support from Earth.

To help deal with such situations, the European Space Agency is in the process of developing CAMDASS, the Computer Assisted Medical Diagnosis and Surgery System.

Utilising an augmented reality headset CAMDASS precisely combines computer-generated graphics with the wearer’s view. Thus, the medical expert on a mission can don the headset and carry out a procedure, precisely guided by a 3D “overlay” and images presented on screens above the operating table. CAMDASS currently uses ultrasound, already use as an examination / diagnosis tool on the space station, but ESA note it could be enhanced and used in surgical procedures.

CAMDASS is a wearable augmented reality headset which could be used to guide a medical expert on Mars through a procedure they've never previously performed
CAMDASS is a wearable augmented reality headset which could be used to guide a medical expert on Mars through a procedure they’ve never previously performed

A final intriguing aspect of the use of VR and virtual environments in a humans to Mars mission is related to public engagement in such a mission.

We might not be able to all go to Mars ourselves, but  as Richard Terrile, director of the Centre for Evolutionary Computation and Automated Design at NASA’s Jet Propulsion Laboratory noted in 2012, virtual environments using data and footage returned from Mars during a mission, could allow “Earthbound humanity to take the journey as well and sense the thrill of walking on an alien world. Three-dimensional surface models with real physical properties and used with avatars further allow the simulation of surface deformation. Thereby, the public can virtually explore, dig holes, move rocks and leave foot prints on Mars.”

VR and vw technologies might even allow those on Earth share in the experience of standing on Mars and exploring that world (image: Explore Mars)
VR and VW technologies might even allow those on Earth share in the experience of standing on Mars and exploring that world (image: Explore Mars)

We’re still most likely at least two decades away from seeing humans on Mars, even allowing for ambitious plans such as Dutch-based Mars One (where the use of VR and VW environments might be even more vital for crew welfare). Obviously, in that time, VR, AR and virtual worlds are going to mature enormously – and so is their likely relevance for long duration space missions as well. As such, what is presented here merely scratches the surface. It’s going to be an interesting future!

One thought on “Why we’ll take VR and virtual worlds to other planets

Comments are closed.