Lab Gab summary: Grumpity, Mojo & Patch – SL Mobile, land, bots & more

via Linden Lab
On Friday, March 10th, 2023, Linden Lab streamed a special session of Lab Gab featuring the Office of Second Life: Grumpity Linden, Mojo Linden and Patch Linden.

The session was built around announcements and updates relating to various initiatives and product developments. Note that the following is not a direct transcript of the entire session, but is intended to record the key points discussed.

For ease of reference, timestamps are provided to the relative points within the video where specific topics are discussed, allowing readers who prefer to listen to the comment directly to be able to do so.

Table of Contents

A Little Background

Grumpity Linden: heads up Second Life Product team, where she has overseen a shift to growth, a stronger, more balanced economy, movement towards better community cohesion, and an overall forward-looking approach. She originally started at LL whilst working for The Product Engine, and was involved in the development of Viewer 2 (as designed by 80/20 Studio), prior to joining LL full-time in 2014. As the Vice President of Product, she is responsible for coordinating the various teams involved in bringing features and updates to Second Life, liaising with legal, financial and compliance to ensure features and capabilities meet any specific requirements in those areas, and so on; work which can involve looking at specifics within various elements of the overall SL product, such as UI design and layout, etc.

Mojo Linden: joined the Lab in 2021 at the Vice President of Engineering, filling the shoes worn for so long by Oz Linden. A 20-year veteran of the gaming industry, he has been responsible for launching numerous games across multiple genres and platforms, and has a strong understanding of platforms, architectures, and product development and technical capabilities. In his role at the Lab, he has shown enormous openness and candour in seeking to increase the platform’s functionality and performance, and in pushing to expand SL’s capabilities.

Patch Linden: originally a Second Life resident and business owner who joined the platform in 2004, and became a Linden in September 2007. He worked across a number of teams within the company – notably within the support and product spheres, and is responsible for developing the Land Operations team, and more recently setting-up the company’s support office in Atlanta, Georgia. Since 2019, he has been Vice President, Product Operations.

Second Life Mobile

[Video 0:00-4:42 and 6:36-7:57]

Grumpity Linden holds up a mobile ‘phone showing Second Life running on it using the upcoming SL Mobile capability
  • Built upon Unity.
  • Core development of this product began in October 2022, with the aim of gaining as high a fidelity of experience on mobile as is possible.
  • Works on all platforms – Android, iOS, tablets, ‘phones.
  • The product is still very much in development, with the focus having been on graphics fidelity, including full avatar rendering.
  • There is some way to go before the product is ready for release – work at the moment is on smoothing out some of the rendering edge cases before moving on to adding further functionality.
  • Initial release is planned for late 2023 with limited functionality, which will then be iterated upon to add more and more functionality and capabilities.

Land Pricing and Fees

[Video: 8:59-12:06]

  • See: Infrastructure Investment Update: Buy/Sell Fee Change and Land Pricing Effective Mar 6, 2023 – official blog post.
  • Grumpity re-iterated the drive (initiated by the late Ebbe Altberg, not long after he joined LL as CEO in 2014) to lower the cost of virtual land in SL – as had long been called for by SL residents – whilst also offsetting the lower land tiers through increases in fees elsewhere within the platform.
  • The fee increases  – as noted in the official blog post – also relate to increased costs LL has faced, thanks to the general economic situation.
  • The fact that the option for paying for land using L$ (restricted to Premium Plus subscribers and limited to one region only at the time of writing) is higher than the USD fee for the same is explained as being a combination of the facts that the L$ to USD exchange rate, the fact that LL have to do the lifting of conversion from L$ to LSD through the LindeX, and as such must participate in the SL economy.
  • It was also noted that paying in L$ means subscribers avoid being charged VAT / state sales tax / similar.

Bots and Policies

[Video: 12:09-13:05]

  • Most likely raised in relation to ongoing concerns about the amount of data being gathered by the BonnieBot system (which has done much to raise the visibility of bots (aka Scripted Agents) in general), and which has also seen some heavy-handed forum moderation on the part of LL. So details can be found threads  – see Has anyone else noticed the B[xxxx] accounts? Sorry. This is not OK. B bots profile scraping and Questions for the Lindens.
  • Whilst the BonnieBot website has take steps away from publishing information which might be regarded as being within Second Life under the expectation it is private (a point that Linden Lab seemingly missed in their initial responses to forum thread postings), the Lab has recognised the need for clearer / better policies and has promised to provide them.
  • These new policies have gone through the necessary compliance and legal reviews within Linden Lab, and will hopefully be published in week #11 (commencing Monday, March 13th, 2023).
  • In addition, a new set of estate / region controls are being developed to enable estate / region holders to better control the use of Scripted Agents within their land. These will be deployed “soon” – hopefully within weeks of this session being aired.

PBR – Physically Based Rendering and Graphics

[Video: 13:10-18:00]

  • (Note: this topic is also covered within my Content Creation User Group meeting summaries.)
  • In brief: PBR essentially models the flow of light within computer graphics in a manner which mimics light reflections on surfaces in the physical world.
  • It represents a significant technical step forward in graphical rendering for SL (in the order of 10 years).
  • The core of this work is to support a new approach to material maps / surfaces either directly on products development for SL by content creators or which can be used on suitable objects within Second Life, which give they sense of physical world lighting and reflection.
  • In particular, it draws on the Khronos glTF 2.0 specification, and allows creators to more readily use a range of tools supporting that standard (including Blender) and also contents libraries, etc., in support of their work.
An Intel-developed scene imported by Nagachief Darkstone and WindowsCE to demonstrate reflection probes (note the reflections on the knight’s armour – these are not generated by attached environment lights but by a reflection probe within the building structure. Image courtesy of Rye Cogtail
  • Also included in the PBR work is reflection probes – which in simple terms enable light sources to generate reflections of an in-world scene on suitable surfaces (i.e. those using the materials system).
  • None of this work does not necessarily mean users need higher specification computers in order to view it; however, reflection probes can impact system performance, so the viewer will include an option to enable / disable these if they prove to be a performance hit for people.
  • Demos of the PBR work, including an imported Sponza scene, can be found on the following regions on Aditi (the Beta grid):  Materials1; Materials Adult and Rumpus Room 1 through 4.
  • The PBR viewer itself – required to see the content on Aditi, is (at the time of writing) still in the Project Viewer stage (so not ready for use as a primary viewer!) and is available through the Alternate Viewers page.
  • The next stage of the project is to provide support on Agni (the main grid) and promote the viewer to Release Candidate status, both for more widespread testing, but there is no set time frame for this to happen.
  • This work does not mean LL are investing in a new graphics engine – this work, and the recent performance improvements work has been on updating and enhancing the existing graphics engine to better support users and what might be regarded as recognised graphics standards.

Mirrors in SL

[Video: 16:23-17:26]

  • The PBR work takes SL some way towards provisioning real-time planar (that is, flat) mirrors.
  • Once the initial PBR work has been deployed, LL plans to start work on offering planar mirrors in some capacity.
  • Further details on this can be found in my 2023 week 7: SL CCUG and TPVD meeting summaries: Mirrors! article.

Graphics API

[Video: 17:27-18:00]

  • While the graphics rendering engine is not being changed, per the above notes, the graphics API will be changing.
  • As OpenGL is going increasingly old (and has been deprecated by Apple), plans are in hand to move the Graphics API to Vulkan (and MoltenVK for Mac).
  • This switch will further enhance SL’s graphics capabilities, and work on the switch-over should commence later in 2023.

Viewer Updates

[Video: 18:00-19:50]

Inventory Updates

  • (Note: Updates on the progress of the Inventory work can generally be found in my TPV Developer meeting summaries.)
  • Work is in progress to provide updates to Inventory management, some of which are designed to help with avatar customisation.
  • The first part of this is to provide an “Inventory thumbnails” capability – offering a thumbnail image of items in Inventory.
  • Alongside of this will be a single folder Inventory view.
  • These are not seen as major gamer changers, but rather quality of life improvements for new and existing users.

Emojis

  • LL is still working on integration the Catznip Emoji code contribution to all emojis in chat and IMs.
  • A project viewer for this should be available in “weeks”.

Linux Support

[Video: 20:48-21:30]

  • The current stop-gap for Linux is running the Windows viewer through Wine or Proton.
  • The need to provide direct Linux support is recognised, but updating the tools to make this possible is not on the roadmap for 2023.

SL and VR

[Video 19:56-20:47]

  • VR still requires a consistent 60 fps left eye / right eye), and for many; the feeling at LL is that  SL is not there yet even with all the recent performance improvements.
  • Various initiatives – adoption of Vulkan, the Puppetry Project – should help improve things further,  and LL might reconsider VR headset support in 2024.

Marketplace Updates

[Video: 21:33-24:52]

  • Marketplace Search was updated with significant changes to ElasticSearch (the underpinning Second Life and Marketplace Search) – see: We’re Improving Marketplace Search
  • This was a large update (lifting the search engine from version 2.3 to 8.4, and did involve some issues on deployment, which are being addressed – see: Updates to Marketplace Search.
  • Further fixes and improvements to the update will be deployed in due course, the first of which will be Boolean searches.
  • A fix for Featured Lists not updating correctly is also due to be deployed “soon”.

New User Experience (NUX) Mesh Avatar

[Video: 25:49-27:46]

A preview version of the new, single mesh (head-to-toe) avatar, which will use by mesh clothing and Bakes on Mesh, and be open for creators to design clothing and accessories
  • That said, the project has taken longer to develop than had been anticipated.
  • There will be a dev kit for creators wishing to support the NUX Avatars. This will be launched ahead of the release of the new avatars.
  • The entire system is intended to be “PBR ready” in that PBR materials may not be included in the clothing and accessories released by the Lab, but the items will be capable of supporting the use of PBR materials on them.

Linden Homes Update

[Video: 27:30-28:09]

  • A full update will be coming soon.
  • The first range of Premium Plus Linden Homes is nearing completion.
  • Updates and expansions to the current Premium Linden Homes – like the Victorian – are being prepared.

Second Life User Group Meetings

[Video: 24:54-25:47]

Closing Comments

[Video 28:10-End]

Everyone is pretty much having fun with the SL Mobile project, and hopes were expressed that a “blooper” reel could be made available. It was also indicated that news on the theme for the 20th Second Life Anniversary event (SL20B) will be made available in due course.

2023 SL Puppetry project week #10 summary

Puppetry demonstration via Linden Lab – see below.  Demos video with the LL comment “We have some basic things working with a webcam and Second Life but there’s more to do before it’s as animated as we want.”

The following notes have been taken from chat logs and audio recording of the Thursday, March 9th, 2023 Puppetry Project meetings held at the Castelet Puppetry Theatre on Aditi. These meetings are generally held on alternate weeks to the Content Creation User Group (CCUG), on same day / time (Thursdays at 13:00 SLT).

Notes in these summaries are not intended to be a full transcript of every meeting, but to highlight project progress / major topics of discussion.

Project Summary

General Project Description as Originally Conceived

LL’s renewed interest in puppetry was primarily instigated by Philip joining LL as official advisor, and so it really was about streaming mocap. That is what Philip was interested in and why we started looking at it again. However since Puppetry’s announcement what I’ve been hearing from many SL Residents is: what they really want from “puppetry” is more physicality of the avatar in-world: picking up objects, holding hands, higher fidelity collisions. 
As a result, that is what I’ve been contemplating: how to improve the control and physicality of the the avatar. Can that be the new improved direction of the Puppetry project? How to do it?

Leviathan Linden

  • Previously referred to as “avatar expressiveness”, Puppetry is intended to provide a means by which avatars can mimic physical world actions by their owners (e.g. head, hand, arm movements) through tools such as a webcam and using technologies like inverse kinematics (IK) and the  LLSD Event API Plug-in (LEAP) system.
    • Note that facial expressions and finger movements are not currently enabled.
    • Most movement is in the 2D plain (e.g., hand movements from side-to-side but not forward / back), due to limitations with things like depth of field tracking through a webcam, which has yet to be addressed.
  • The back-end support for the capability is only available on Aditi (the Beta grid) and within the following regions: Bunraku, Marionette, and Castelet.
  • Puppetry requires the use of a dedicated viewer, the Project Puppetry viewer, available through the official Second Life Alternate Viewers page.
  • No other special needs beyond the project viewer are required to “see” Puppetry animations. However, to use the capability to animate your own avatar and broadcast the results, requires additional work – refer to the links below.
  • There is a Puppetry Discord channel – those wishing to join it should contact members of LL’s puppetry team, e.g. Aura Linden, Simon Linden, Rider Linden, Leviathan Linden (not a full list of names at this time – my apologies to those involved whom I have missed).

Additional Work Not Originally In-Scope

  • Direct avatar / object / avatar-avatar interactions (“picking up” an apple; high-fives. etc.
  • Animations streaming: allowing one viewer to run animations and have them sent via the simulator to all receiving viewers without any further processing of the animations by those viewers.
  • Enhanced LSL integration for animation control.
  • Adoption of better animation standards – possibly glTF.
  • Given the project is incorporating a lot of additional ideas, it is likely to evolve into a rolling development, with immediate targets for development / implementation decided as they are agreed upon, to be followed by future enhancements. As such, much of what goes into the meetings at present is general discussion and recommendations for consideration, rather than confirmed lines o development.

Bugs, Feature Requests and Code Submissions

  • For those experimenting with Puppetry, Jiras (bug reports / fixes or feature requests) should be filed with “[Puppetry]” at the start of the Jira title.
  • There is also a public facing Kanban board with public issues.
  • Those wishing to submit code (plug-ins or other) or who wish to offer a specific feature that might be used with Puppetry should:

Further Information

Meeting Notes

Viewer Progress

  • An updated version of the project viewer is due to be made available once it has cleared LL’s QA process. This includes:
    • Using the binary protocol for the LEAP module communication, with new logic which causes LEAP modules to one be loaded by the viewer when they are used.
    • The AgentIO LEAP module adds the ability to adjust the look at target, viewer camera and agent orientation.
    • Support for sending the joint position of your avatar to the server, which is then available in LSL.
      • The code reports the post animation location for attachment points, allowing the the sever to know where things like hands and wings, etc.,  are, and this in turn allows LSL to query where that attachment point is in space and how it is rotated.
  • HOWEVER, the animation streaming code (see previous Puppetry meeting notes) will not be in the next viewer update.

Server-Side Work

  • The simulator code now has llGetAttachmentPointAnim() support, which should be recognised by the upcoming viewer update.
  • The Aditi puppetry regions are to be merged with the updated code so this can be tested.
  • While there has been some work completed on animation imports since the last meeting, there was nothing significant for LL to report on progress at this meeting.

General Notes

  • There is additional work going on to try to improve the IK system, with the aim of having the basics working better than is currently the case – better stability, etc. This work may appear in the viewer update after the one currently being prepared to go public.
  • Performance:
    • To prevent puppetry generating too much messaging traffic (UDP) between the viewer and simulator, a throttle is being worked on so that when the simulator is under a heavy load from multiple viewers running puppetry code, it can tell them all to tone down the volume of messages.
    • There will also be some switches and logic put into place that can be used when needed, helping to protect regions in case the load gets overwhelming.
    • A further suggestion made is to ensure the simulator does not broadcast puppetry messages for avatars seated and not using the code (such as an audience at a performance) to further reduce to volume of messaging, this is viewed as a potentially good avenue of work to consider.
    • There is also a threshold in place – if an attachment point does not move beyond it, it is not considered as moved, which will hopefully also reduce the amount of messaging the simulator has to handle.
  • LSL Integration:
    • See: OPEN-375: “LSL Functions for reading avatar animation positions”.
    • This work is now paused. Rider Linden developed a proof of concept, but found that in order to better manipulate parameters within the constraints, a configuration file should be used. He is therefore refactoring the code to do this before proceeding further.
    • The configuration file will be called avatar_constraints.llsd and it will live alongside avatar_lad.xml in the character directory.
  • Questions were again raised on whether Puppetry is for VR / will enable the viewer to run VR.
    • It was again pointed out that while Puppetry lays more foundational work which could be leveraged for use with VR headsets, than is not the aim of the Puppetry project.
    • Providing VR headset support is a much broader issue, which would require the involvement of other teams from LL – Product, the Graphics Team, the viewer developers, etc.

Date of Next Meeting

  • Thursday, March 23rd, 2023, 13:00 SLT.