Behind the scenes with Bento in Second Life

Bento: extending the avatar skeleton
Bento: extending the avatar skeleton

Note: this is a compressed version of an article which first appeared in Kultivate Magazine’s January 2017 issue, and which here includes some personal feedback on being peripherally involved in the project.

There can now be few – even among those having only joined Second Life in the last few months – who cannot be aware of project Bento, the work to significantly enhance the Second Life avatar skeleton to better support mesh avatar models, how they are animated and how they might be customised. I was able to observe Bento almost throughout its lifespan. As such, I thought more of a look inside the project might be of interest.

Bento’s heart comes in the form of one person: Vir Linden. Despite the avatar skeleton being one of the most fundamental parts of Second Life which could if interfered with, cause all sorts of issues, Vir was convinced things could be done to make the lot of mesh avatar creators and animators much easier, and without putting additional stress on either the simulator or the viewer. So in early 2015, he started looking at what might be achievable.

Vir Linden, Senior Software Engineer at Linden Lab, and technical project lead for Bento
Vir Linden, Senior Software Engineer at Linden Lab, and technical project lead for Bento

By roughly April 2015 he felt he had enough information to put together a presentation he could put before senior staff at the Lab. Despite the ambitious nature of the project, potentially adding (at that time) 106 new bones to the avatar, the response was enthusiastic and positive. Troy Linden joined Vir from the Product Team, and Bento was born.

Initially, the work was in-house, involving Lab staff and the Moles. However, as the potential  for the project grew, it quickly became apparent that wider expertise would be needed to both give input to the project and help steer the development work. And so it was that a core Bento User Group came together, comprising Lab staff with expertise with the viewer, with the server, and – most significantly – content creators noted for their expertise in developing mesh avatars and avatar accessories (human and non-human), and the people behind the most popular tools for avatar creation and animation: Gaia Clary and Matrice Laville (Avastar) and Cathy Foil (Mayastar).

Like, Vir, Cathy, Gaia and Matrice deserve special recognition for their work on Bento. Not only did they embrace the project and work to update their products to support Bento, they also took on a lot of the grunt work involved in updating essential code used by the viewer to make Bento happen – and they were enthusiastic about doing so. If you make use of a Bento head and are adjusting things using the sliders, offer a word of thanks to the three of them: they’re the people who carefully went through the avatar files in the viewer to get as many of the sliders as possible working with the new facial bones for you to be able to do so.

Vir linden (foreground) and Matrice Laville at a Bento project meeting
Vir Linden (foreground) and Matrice Laville (in the top hat, left) at one of the closed user group Bento project meetings (note the fellow in the top hat to the right of the picture is Rider Linden)

Bento meetings initially took place at a small desert island on Agni, the Main grid. This sat on its own channel to ensure the necessary tweaks to the simulator end of things could be made to try out ideas and options and generally lay the foundations of the project. This work progressed over a period of just over six months until there was a good level of confidence that the project could be taken to the next stage.

This involved moving things to Aditi, the beta grid, and opening things to the public at large. The use of Aditi meant that more extensive testing of simulator and viewer changes could take place, more space could easily be made available for people to experiment, and more people in general could get involved. In time, these meetings would eventually move back to the main grid. It also marked the start of my ability to publicly report Bento progress on an almost weekly basis.

Bento enables much more flexibility when creating and animating human and non-human mesh avatars
Bento enables much more flexibility when creating and animating human and non-human mesh avatars

At the start of the public phase of Bento, the skeleton had stabilised with some 92 additional bones – 60 of which were evenly split between hands and head. However, as the work progressed, this number tended to vary for a time as the need for further bones was identified while some of the new bones were found to be redundant and could be removed. As the same time, additional use cases for Bento were proposed, with every effort being made to incorporate as many as practicable. There were even some fundamental shifts made in how Second Life functions so that more could be achieved through Bento.

An example of this lies with bone translations and rotations. In essence the SL animation system had always worked on the assumption that sliders only affect translation and scale whilst animations only affect rotation, allowing both to work cooperatively.As the animation system was not necessarily seen as an intrinsic part of Bento, little thought was initially given to changing it. However, during the public beta phase of the project, a strong case was made for allowing both translation and rotation through animations – and as a result, the necessary updates were made to the animation system to support this without potential conflicts occurring.

Continue reading “Behind the scenes with Bento in Second Life”

SL project updates 2017-1/1: general status

A Painter's Link, Salomon Beach; Inara Pey, December 2016, on Flickr A Painter’s Link, Salomon Beach (closing January 6th, don’t miss!) – blog post

Server Deployments

The RC channels were re-started on December 28th, 2016, to the consternation of some on the Server thread of the technology forum as there was no accompanying update. A rolling restart of the Main (SLS) occurred on Tuesday, January 3rd, 2017, again with further consternation on the thread.

However, as a quick check through the viewer can reveal, both of these operations were simply restarts to keep things running (reasonably) smoothly, as shown by the server version numbers remaining unchanged when viewed in the viewer (Help > About viewer).

No restart is anticipated for the RC channels on Wednesday, January 4th, given they were restarted at the end of December. The usual run of deployments + restarts are expected to resume in week #2 (week commencing Monday, January 9th) with the three RC channels.

SL Viewer

With the holiday break, there has been no movement with the official viewer, with four currently occupying the various pipelines:

  • Current Release version: 5.0.0.321958, dated December 1st, promoted December 5th – formerly the Project Bento RC viewer
  • Maintenance RC viewer, version 5.0.1.322513, dated December 21st – some 42 fixes and improvements + Bento support
  • 360-degree snapshot project viewer, version 4.1.3.321712, dated November 23rd – ability to take 360-degree panoramic images – hands-on review
  • Obsolete platform viewer version 3.7.28.300847, dated May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

It had been indicated that the Lab might get a project version of their 64-bit viewer out prior to the holiday break, but this proved not to be the case.

 

A look at Second Life updates in 2016

A Painter's Link, Salomon Beach; Inara Pey, December 2016, on Flickr A Painter’s Link, Salomon Beachblog post

Each week through the year, I try to get to as many in-world and other meetings held by the Lab to keep an eye on technical developments and updates which are in the works for the viewer and the simulator, relaying the notable items via my SL project updates. As such, I thought it might be interesting to look back at some of the technical changes and updates have come our way in 2016.

The Big Ones

There were obviously a number of fairly high-level updates which came our way, notably Project Bento, which gave us a lot of new bones and attachment points specifically for mesh avatars to make them more flexible and easier to animate. There’s a whole story behind that project which perhaps hasn’t been told in full, so expect to read more from me on it in the New Year 🙂 .

Then there was Avatar Complexity, or Jelly Dolls as it has been more popularly dubbed (initially by Whirly Fizzle after it was pointed out the term “Jelly Baby”, as initially used was in fact a trademark). Avatar Complexity is designed to reduce the often high cost of avatar rendering by the viewer, thus lightening the load on computers / graphics cards which might otherwise struggle.

A longer-term hope may have been that perhaps it would encourage people to consider what they are wearing and how it may affect others, and even get content creators to think more conservatively about their creations, and seek to optimise them for rendering. Whether either of these latter points might be / already have come about is nigh-on impossible to judge.

The Viewer

Both Project Bento and Avatar Complexity involved some pretty substantial changes to the viewer – Bento to the degree it warranted a version number boost. But they weren’t the only significant changes. There were also 6 new Maintenance viewers through the year, bringing with them over 250 fixes, updates and improvements. Besides these the following notable viewer releases / updates also appeared through the year.

Graphics Presets

Alongside of Avatar Complexity we gained Graphics Presets, another useful means to help improve viewer performance by allowing users to save different graphics set-ups for the viewer. This means, for example, we can have a preset for taking photographs, with all the more taxing graphics options – shadows, lighting, longer draw distance, etc – can be enabled, then have another for, say, shopping, where all the bells and whistles aren’t required, helping to improve viewer performance – and we can quickly change between them without all that tedious mucking about in Preferences.

Graphics Presets (also in most TPVs) allows you to create and save your own graphics presets to suit different requirements, and which can be quickly loaded and used with just a couple of mouse clicks
Graphics Presets (also in most TPVs) allows you to create and save your own graphics presets to suit different requirements, and which can be quickly loaded and used with just a couple of mouse clicks

If you are on a system which can struggle at times because you have your graphics settings tweaked a little on the high side and you’ve not experimented with Graphics Presets, you might want to give them a try.

Visual Outfits Browser

The Visual Outfits Browser brought with it the ability to have images associated with your Outfits (if you use the Outfits capability). Feedback on this seems to have been mixed. Many like it, while many, equally, ignore it (and I’m among the latter category).

HTTP Updates

The viewer received a lot of new under-the-hood HTTP updates, including the removal of  a considerable amount of deprecated and unused code, and a series of improvements for things like image, mesh and animation uploads, inventory manipulation, the Viewer Management Marketplace, LSL script compilation, Experiences management, etc.

Voice Updates

Voice has been worked on throughout 2016, with the Lab working closely with the Voice package provider, Vivox, to improve connectivity, overcome Voice quality issues, and removed many of the known exploits as possible to prevent thinks like a user in one region eavesdropping on a conversation being held in another region.

This work has involved changes to the viewer, changes to the simulator, changes to the Voice binary package supplied by Vivox (SLVoice.exe) and even changes to the Vivox servers (Voice is routed through their own servers).

LibVLC

As Apple dumped QuickTime for Windows with potential security vulnerabilities unpatched, The Lab adopted LibVLC for media handling in the Windows viewer (and will be moving to it win the Mac and Linux viewers when their have released their 64-bit viewers).  The move overcomes most issues in trying to play back media in-world, however, licensing around the Advanced Audio Coding and MP3 formats, and the way things are packaged with LibVLC might leave TPVs with a headache or two.

Inventory Handling

Aura Linden worked on removing deprecated and unused UDP inventory messaging mechanisms from the viewer. This work is to be followed by the removal of back-end support for the removed message channels, and further viewer-side work on rationalising and refactoring the code handling inventory operations.

360 Snapshot Viewer

Whilst still only a project viewer, the 360 snapshot  viewer is part of a viewer / simulator project to bring 360-dgree photography to Second Life.

Linux

One unpopular move was the announcement concerning Linux development going forward (although the Lab will be building a 64-bit Linux viewer).

The Simulator and Servers

The simulator software continued through its weekly deployments throughout the year, added bug fixes, security updates, feature requests and more each month. Listing everything that happened here would rapidly turn this article into a TL;DR. However, as well as the continued deployment of simulator code updates, 2016 saw the mechanism and tools used to build the simulator undergo update, as was (/is) the underpinning server operating system running the simulators.

Support for larger animation files was introduced, with uploads increased from 120Kb to 250Kb.

Group bans finally got a tweak so that those banned from a group whilst active in group chat would finally get booted from the group chat session as well.

Experiences got a new scripted sit capability, code-named Project Espeon.

Experience scripted sits came our way in 2016 (image courtesy of Linden Lab)
Experience scripted sits came our way in 2016 (image courtesy of Linden Lab)

And, of course, we have the increases to Land Capacity (or LI or prims, however you like to think of it).

Aditi Inventory Syncing

A new process for syncing inventory between Agni (the Main grid) and Aditi (the beta grid) was introduced, eliminating the need for users wishing to have their Agni inventory fully replicated on Aditi having to change their SL password and then wait between 24 and 48 hours (sometimes longer) for their Aditi inventory to be synced with Agni. Under the new system, a process automatically merges a copy of users’ Agni inventories with their Aditi inventory based on their last log-in to the beta grid.

There were some teething problems with the new system when first introduced (and some people report there may still be hiccups), but on the whole the new process is a lot smoother than the old.

Web Services: TLS 1.2 and More

The Lab made the switch to TLS 1.2, which had the potential to impact people’s ability to buy L$ via the LindeX / through a browser and / or add payment info to their account if they were not using a suitable viewer or web browser.

There were also numerous changes to various web properties, including updates to the SL Marketplace, the retirement of SLurl.com, various security and infrastructure updates

Grid Status Page

The Grid Status page moved to a new provider and was overhauled to be hopefully more informative, and have a faster means of update.

 2017 Expectations

The lab plays their cards close to their chests when talking about upcoming changes / updates / improvements to Second Life, but here’s a (short) list of some of the things we can reasonably expect to see in 2017:

  • 64-bit versions of the official viewer (Windows, Mac and Linux).
  • Possible changes / tweaks to the avatar / object complexity calculations made by the viewer, such as it being able to more easily determine those avatars in its field of view it should not attempt to fully render (rather than waiting on information from the simulator to make that determination).
  • Further updates to the viewer build tools (e.g. VS 2015 for Windows).
  • Progress on the 360 snapshot viewer.
  • Further work cleaning-up and rationalising the viewer code.
  • Voice updates for both the server and the viewer.
  • Continued server deployments and improvements 🙂 .

You can follow my updates on SL technical developments and updates through the likes of my weekly SL project updates and weekly viewer release summaries (which also cover TPV releases).

SL project updates 2016 51: server, simulator OS update

DRD Arctic Express
DRD Arctic Expressblog post

Server Deployment

While the No Change window was supposed to have come into operation on Friday, December 16th, there was indeed a deployment to the Main (SLS) channel on Tuesday, December 20th. It comprised the same server maintenance package as deployed to the RC channel in week #50, comprising internal logging changes.

The deployment means that all of the server channels on the main grid (Agni) are now running the same simulator version. As there are no planned deployments to the RC channels, all four channels should remain on this release until deployments resume after the holiday period.

SL Viewer

We might see a 64-bit project viewer appear during the week. However, at the time of writing,  the list of viewers in the various pipelines remains as:

  • Current Release version: 5.0.0.321958, dated December 1st, promoted December 5th – formerly the Project Bento RC viewer.
  • Release channel cohorts:
    • Maintenance RC viewer, version 5.0.1.322219, dated December 9th – some 42 fixes and improvements + Bento support
  • Project viewers:
    • 360-degree snapshot viewer, version 4.1.3.321712 dated November 23rd – ability to take 360-degree panoramic images – hands-on review
  • Obsolete platform viewer version 3.7.28.300847 dated May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

Bento Support

With the release of Kokua for Second Life 5.0.0 and RLV 2.9.21 (see my article here), the following currently maintained viewers all have Bento support: Black Dragon, Catznip, Cool VL, Firestorm, Kokua for Second Life, Restrained Love. Alchemy is expected to update to include Bento “soon” and UKanDo is still awaiting a complete overhaul.

In terms of clients with 3D viewing capabilities, Radegast has been updated to support Bento, but the update is pending release. Lumiya for Android will have Bento support added “soon” (Lumiya 3.3 with Voice support released on Sunday, December 18th, and my review is available here).

Simulator OS Update

The Lab is in the process of updating the operating system on the simulator servers. At the moment the new OS version is installed on a number of regions on Aditi (the beta grid), including: Fire Ants 1, Fire Ants 2, Grasmere, Oak Forest, and Twilight Shores, not all of which are open to the public.

“This is one of those ‘features’ that might perform a bit better, but otherwise is a success if it behaves exactly like our current servers,” Simon Linden said at the Simulator User Group meeting on Tuesday, December 20th.

“We’re in the starting process of testing things. We have a lot of testing to do. [We’re] updating from an old version of Linux to a not-so-old version of Linux,” April Linden added. “Just to keep things moving. All the usual reasons. Performance updates, security, etc.”

Second Life hits a lot of low-level server code hard, including networking, memory and multi-processing. The overall hope is to have the newer revisions of the operating system will improve these. Expect this work to reach Agni in early 2017.

Group Chat

Group chat was long a bane of Second Life. In 2014/15 however, the Lab put a considerable amount of work into improving things, although some issues remain. One of these is that if an individual role outside of the default “Everyone” has a very high number of group members assigned to it (e.g. several thousand), it can dramatically impact things like group chat performance and can prevent the members’ list loading. One solution might be to delete the role (converting those in it to Everyone).

SL project updates 2016 50/3: TPV Developer meeting

Mineral Ridge; Inara Pey, December 2016, on Flickr Mineral Ridgeblog post

The notes in this update are taken from the abbreviated TPV Developer meeting held on Friday, December 16th. The video of that meeting is embedded at the end of this update. My thanks as always to North for recording and providing it.

SL Viewer

Current Pipeline

Expect no further viewer promotions to release or RC status until 2017, as the No Change window is now in effect through until January 2nd, 2017. although this doesn’t necessarily prevent project viewers from appearing (see below).

64-bit Builds

[00:07] The Lab is progressing with the 64-bit builds (Project Alex Ivy – which I assume is a reference to 64 in Roman numerals: LXIV = aLeX IVy). Oz reports they are not quite there with the Mac builds as yet, with more work needed on the library builds. However, Whirly Fizzle has already uncovered  some issues with the Windows build, although she reports it as being largely stable through her own testing. Oz indicates there is still “some chance” a project viewer may appear before the holiday break takes full hold.

360-Snapshot Viewer

[05:59] The 360 snapshot viewer remains on hold while development is focused is on the 64-bit viewer builds. As soon as working versions of the latter are available for all the OS platforms, efforts will be switched back to 360-snapshots.

Viewer-side Voice Updates

[01:54] A new Voice update viewer should be available early in the New Year. This should fix a number of bugs and add improved diagnostics. There is currently no time line on when older versions of the Voice package will be blocked from connecting to the service, as “other things” are now seen as having a higher priority.

New Strategy for Rendering Fixes

[16:45] Up until now, fixing for rendering issues have been handled as a part of the Lab’s “standard” methodology for viewer updates: develop a fix for an issue and release it in a Maintenance release candidate viewer. This hasn’t always worked, with some fixes introducing problems of their own (the solid grey / black rendering of invisiprims perhaps being the most visible in recent times), which then prevent other fixes in the same Maintenance RC from progressing while the Lab works on the rendering fixes.

To try to avoid this, the Lab is going to experiment with separating out rendering fixes and moving them to their own release branch of the viewer. This should both prevent other Maintenance fixes and updates from being bottlenecked, but also allow for better QA testing of rendering system fixes / changes.

Sounds and Animations: HTTP and CDN Delivery

[02:22] In 2013 / 2014, the Lab made a huge change to how avatar appearance information and texture and mesh assets are delivered to users, shifting them away from UDP (User Datagram Protocol) delivery through the simulators, to HTTP via Content Delivery Networks (CDNs) – see my past reports on the HTTP updates. and CDN work.

For 2017, the Lab plan to move sound and animation assets (which may or may not include gestures) to delivery via HTTP and thence to CDN distribution and delivery. There is no precise time frame for this work, but once fully implemented (including by TPVs) the UDP  / simulator messaging and routing for these assets will be removed.

While some have experienced CDN related issues with textures and meshes, the hope is that the move will make the delivery of sounds and animations more robust and faster (sounds are reported as being “amazingly slow” to delivery over UDP via the simulator – 10K/second), and remove more of the heavy lifting of assets from the simulators.

Other Items

First TPV Developer Meeting for 2017

[24:26] The December 16th meeting marked the last TPV Developer meeting for 2016. The next meeting will not by until Friday, January 13th, 2017.  This is largely due to the fact that not a lot will have changed between now and the second week of January to make a meeting worthwhile.

Second Life and Oculus Rift

[33:28] In July 2016, Linden Lab suspended development on Oculus Rift support in the viewer, but left the door open a crack for the potential for the work to be picked-up at some point in the future. However, as things stand with the current generation of headsets, this is not going to happen in the foreseeable future.

The belief is that the rendering requirements – particular frame rates sit well above those which can be reasonably achieved in Second Life through the viewer (Oculus VR quotes a minimum of 60 fps and a preferred rate of 90 fps, which the Lab sees as being doubled to 120 and 180 fps when rendering an SL scene in stereo).

Firestorm 5.0.1 Download Issues

[20:16] Jessica Lyon gave further insight into the recent issues with people trying to obtain the Firestorm 5.0.1 Bento release (see my review here, and subsequent updates on the download situation here and here).

While there were some indications demand would be high – the Firestorm Preview group had expanded to over 8,000 people – it nevertheless outstripped all expectations, and the Firestorm download server almost came to a standstill. To try to correct this, the server was restarted, driving traffic back to the web server in the process, which then overloaded and crashed.

Firestorm hope that with Bento out the door, the viewer can resume a more “normal” QA / release cycle through the Preview and Beta groups. However, a mirror site for new release downloads will be maintained going forward. Options for hosting the Firestorm JIRA service, considered a major resource hog on the FS servers, are also being considered, although there are pro and cons involved in making changes.

Jess also took the opportunity to again thank Linden Lab for stepping up and providing a mirror for downloads.

SL project updates 2016 50/2: re-caps and Content Creation UG

Fris' Land, Liebe; Inara Pey, December 2016, on Flickr Fris’ Land, Liebeblog post

Server Deployments – Recap

As always, please refer to the server deployment thread for the latest updates and information.

  • On Tuesday, December 13th, the Main (SLS) channel received the same server maintenance package, as deployed to the RC channels in week #49. This includes the following feature requests: BUG-6377 – llGetObjectDetails(id,[OBJECT_ATTACHED_SLOTS_AVAILABLE])  and BUG-40871 – llGetEnv() constant “region_object_bonus”.
  • On Wednesday, December 14th, all three RC channels received the same new server maintenance package, comprising improved internal server logging.

SL Viewer

The official viewers list remains unchanged from earlier in the week:

  • Current release viewer, version 5.0.0.321958, dated December 1, promoted December 5 – formerly the Project Bento RC viewer download page, release notes
  • Maintenance RC viewer, version 5.0.1.322219, dated December 9th
  • 360-degree snapshot viewer, version 4.1.3.321712, dated November 23rd.
  • Obsolete platform viewer, version 3.7.28.300847, dated May 8th, 2015 – provided for users on Windows XP and OS X versions below 10.7.

No Change Window

A reminder that the end-of-year No Change window comes into effect from Friday, December 16th. There will be no further planned server deployments or official viewer promotions after that date, through until Monday, January 2nd, 2017.

Content Creation User Group

On Thursday, December 15th, the Project Bento User Group officially morphed into the Content Creators User Group (thus in a way  also now standing as a revamp of Nyx Linden’s old group).

Meetings will continue to take place at 13:00 SLT on Thursdays, at the Hippotropolis camp fire circle, with the exceptions of Thursday, December 29th and Thursday, January 5th. Anything related to Bento remains on topic for the group, but things of general interest to content creators are also open for discussion.

New Bento Issues

Bento is generating considerable interest among users – as witnessed by the release of Firestorm 5.0.1, when demand overwhelmed the Firestorm servers, and additional mirror services had to be set-up, with assistance from the Lab in doing so. However, some further issues have been identified as a result of Bento reaching a broader audience:

  • BUG-41063 – [Bento] Some Fitted Mesh No Longer Fits on Bento Avatars
  • BUG-41048 – [Bento] When wearing some Bento content, all walking animations freeze

The first of these bugs has been accepted by the Lab and is being investigated. The second appears to be most notably related to animations using the Bento wing bones, which appear to be overriding animation walks, causing an avatar to slide along the ground, rather than walking.

The issue particularly seems to impact walk animation using the llSetAnimationOverride capability, while older ZHAO systems (which tend to use the llPlayAnimation capability, appear to be unaffected. This may be due to the manner in which the latter handle start / stop animations calls, which is successfully preventing the issue manifesting with them.

The underlying cause is thought to be a possible animation priority setting / conflict or possibly the result of the wing bone animations having bone positions being set for all other, but not rotation data – but further investigation is required. Vir has taken samples of items which can cause the issue (again, notably, but not necessarily limited to, wings), and is going to have a poke at them to see if he can see what is going on.

Elsewhere, creators are waiting on Avastar to update, and Damien Fate has a rather neat HUD to help animate Bento hands and fingers. It’s unlisted, so I’m linking to it, rather than sharing it via embedding.

Fun Fact

The new Linden Lab portal parks have a LI capacity of 45,000 (with thanks to Cube Republic and Arton Rotaru for the pointers).

Portal Park LI capacity
Portal Park LI capacity