The Drax Files 41: animating Second Life

The Drax Files World Makers #41 arrived on Wednesday, October 12th, 2016. At just a shade under four minutes in length, it is an intriguing beast, exploring in-world animations and motion capture through a 360-degree video format.

The latter is suitably underplayed at the start of the video, as Drax introduces it – but the clue comes much sooner when watching on a flat monitor – the 360-degree cursor located up in the top left of the screen, which you can use to steer your way around the video view, or you can left click and drag. Obviously, if you’re using a mobile device, you can tilt and turn the device, allowing the gyro to move the image around, and those with a head mounted display can instantly enjoy in in 360-degree surround.

“I’ve been playing with some of the cheaper systems available,” Drax told me, as we discussed the video, and why he opted to go with the 360-degree format for this segment of World Makers. “Like the Ricoh Theta and Samsung Gear 360, and Mambo Morane has been working in real life with the Go Pro array, so I started thinking in June about how we could do this in SL.”

This turned out to be harder than anticipated. The first attempt involved using an array individual viewers synchronised by a device built by Arduenn Schwarztmann which would enable simultaneous filming through all six viewers, and included additional audio cues to further assist in the post-production stitching process. Unfortunately, this approach revealed that differences in how GPUs process the recording, even with the same windlight and camera defaults in the viewer, could result in recorded clips sufficiently different one to another that stitching them together failed to produce a smooth result.

“Then Mambo Morane came up with the idea of filming in six instance of the viewer running on the same machine, using Open Broadcaster Software to bring them all up together,” Drax continued. “We could then pull them apart in post-production and stitch the individual clips together using 360-editing software, with all of them having the same look and feel. Unfortunately, this may not be something for many machinima makers right now. The software for editing and stitching the video cost US $800.”

The result is a very smooth video, freely intermixing physical world footage shot at Vista Animation’s offices near Barcelona, with footage stages and shot in Second Life which presents an exceptionally immersive and unique view of Second Life, even when seen on the flat screen of a video monitor.

Certainly, the 360-degree aspect is guaranteed to be one which will have people watching the video at least twice, simply because scrolling / looking around in side SL is addictive, and there are some nice little touches to be found – such as little Marianne McCann gamely holding up a boom microphone in some of the in-world footage. This inevitably means it is easy to become wrapped-up in scrolling and looking, without paying attention to what is being said, prompting a second viewing to focus on the main aspect of the audio narrative: animations.

Animations – walks, stands, sits, dances, runs, hops, crawls – whatever form they take – are something we’re all familiar with to some degree. An animation override system can often be one of the first purchases made in Second Life (allowing for the worn AOs now supplied with starter avatars and those supplied by the makers of avatars, human or otherwise), and we’re all familiar with the idea of mocap – motion capture – going into their production.

Vista Animations is widely regarded as one of the premier providers of animation packs for overriders, dances, etc., and World Makers #41 offers something of a glimpse into their work, albeit it without going too in-depth with matters of production and workflow (although Drax has previously covered elements of MoCap in Drax Files World Makers #6, so this sits as a good companion piece, and Vista Animations also offer a look at their work for those interested in other aspects of animation creation.

What is offered here is a feel for both the complexity of motion capture and how rapidly the field is changing, as well as a look behind the curtain at a small, successful business which has grown out of Second Life. It terms of the former, the Vista team point out that when they started with their first MoCap suite in 2008, it cost them US $45,000. The system they use today, which I believe was purchased in around 2012/13, set them back US $2,000.

The MoCap process isn’t just a case of pulling on a suit of sensors and then moving around with the cameras running. Everything has to be calibrated – sensors (50+ for the body and additional elements for the hands), skeleton, props, etc. – to ensure a smooth capture process, which can be time-consuming. Then, once captured, there is the entire editing and post-process work required to produce the finished animation files which can be uploaded to Second Life.

While this latter aspect isn’t really touched upon in the video, what is fascinating to see is how physical world actions translate in-world through the clever use of cross-fading in the segment. This is particularly effective as we see Drax doing a mock interview while being motion captured, then transition to him carrying out an interview in-world.

Another interesting aspect to consider here – albeit own mentioned in passing, is that of facial animations.While we’ve had these in SL for as long as we’ve had animations, Project Bento may well revolutionise them, simply because it is giving animators a set of 30 facial bones to play with and animate. As such, it is going to be interesting to see what animators like Vista might come up with (allowing for how closely facial animations may need to be tied to mesh head models).

Similarly (and perhaps more easily) hand motions are the opportunities for finger motion / hand shape capture is liable to be lifted by Bento, given hands will have a total of 30 joints as well. In fact, Vista are already moving in this direction, offering a complete Bento hands system which involves the capture of natural hand / finger movements).

There are many interesting little snippets in this video which can be easily missed when focusing  on the 360-degree aspect of the segment; it was interesting to hear one of the Vista team commenting on the self-conscious feeling he experiences when entering a club and seeing avatars using his dance moves. When you consider the hundreds – thousands, even – of Second Life users who are using these animation, that has to be something of an uncanny valley-esque feeling at times.

The use of 360-degree video in SL is just in is infancy; by contrast, animations are one of the most well-established aspects of the platform. Individually and collectively, they demonstrate how Second life is evolving as an immersive environment, whether it be through the mains by which out avatars can move and behave with a degree of natural flow to their movements and actions, or be it in the many and different ways we can immerse ourselves in the places and spaces Second Life has to offer.

Would Drax be using the 360 video format again?

“Certainly not for the next couple of videos,” he told me. “It’s an intensive process, not just with the video editing, but in editing and matching the spatial audio. But Mambo is continuing to experiment with the approach we used, and might offer a service to business and region owners who might want to have a 360-degree video of their business or sim.

“I’d also like to give a shout-out to Torley Linden, Arduenn Schwarztmann, Pepa Cometa, Erythro Asimov, Rysan Fall and Ole Etzel, all of whom banged their heads with me trying to figure out how to get this to work before Mambo hit on the 6 viewer instance approach.”

And, if you want a free DaxtorPack of animations, featuring the actions of the man himself – head on over to Vista Animations, and pick one up for free! 🙂