Mullein Woods, November 2022 – blog post
The following notes were taken from m y audio recording and chat log transcript of the Content Creation User Group (CCUG) meeting held on Thursday, January 5th 2023 at 13:00 SLT.
These meetings are chaired by Vir Linden, and their dates and times can be obtained from the SL Public Calendar; also note that the following is a summary of the key topics discussed in the meeting and is not intended to be a full transcript of all points raised.
Official Viewers Status
On Wednesday, January 4th, 2023:
The Maintenance Q(uality) RC viewer updated to version 6.6.9.577418.
The Performance Floater / Auto-FPS RC viewer updated to version 6.6.9.577251.
Both the VS 2022 Build RC viewer and the LMR6 project viewer have been withdrawn.
This leaves the rest of the currently-available official viewer as:
Release viewer: Maintenance P (Preferences, Position and Paste) RC viewer version 6.6.8.576863 Monday, December 12, 2022.
Project viewers:
PBR Materials project viewer, version 7.0.0.577157, December 14, 2022. Note: this viewer will only function on the following Aditi (beta grid) regions: Materials1; Materials Adult and Rumpus Room 1 through 4.
Puppetry project viewer, version 6.6.8.576972, December 8, 2022.
Inventory Enhancement Project
Linden Lab is looking to enhance the Inventory system.
The first element of this work is to be the addition of a fixed-resolution thumbnail preview capability, allowing users to see a small image of a given object (where this makes sense – so the likes of note cards and scripts would be excluded) within inventory, with these thumbnails either being of individual items or entire folders.
The first phase of the work is determining how to generate the thumbnail images and ensure they maintain an association with the objects to which they are related (e.g. so if an item is sold or transferred to another user, the thumbnail goes with it).
Once this has been decided, the next phase will be to build-out the UI so that such thumbnails can be viewed from inventory.
This work will not replace the Outfit Folder image capability nor will it prevent creators from including high resolution images with their products if they wish.
Once the thumbnail preview work has been completed, it is possible the Lab will look to further enhancements to inventory management. One future enhancement under consideration is support for folders to be included in the Contents inventory of individual objects.
glTF Materials and Reflection Probes
Project Summary
To provide support for PBR materials using the core glTF 2.0 specification Section 3.9 and using mikkTSpace tangents, including the ability to have PBR Materials assets which can be applied to surfaces and also traded / sold.
To provide support for reflection probes and cubemap reflections.
The overall goal is to provide as much support for the glTF 2.0 specification as possible.
The project viewer is available via the Alternate Viewers page, but will only work on the following regions on Aditi (the Beta grid): Materials1; Materials Adult and Rumpus Room 1 through 4.
The focus remains on bug and regression issue fixing within the viewer and quality of life improvements, particularly as in wider grid testing, it has been found the PBR viewer can only generate single-digit FPS in some regions.
Screen Space Reflections (SSR): Geenz Linden continues to work on integrating SSR into the PBR viewer, but is encountering issues.
Animation System Enhancements – A Discussion
In response to requests for the animation system to be improved (e.g. via CCUG meetings, as a result of the Puppetry project, etc.), Vir Linden asked those at the meeting to state what they see as the most important changes / updates they would like to see. Responses included those expressed at the Server User Group meeting earlier in the week:
A procedural animation system to allow creators / users to set the rules of how avatars walk, run, jump; their timings, how animations play priority wise and mixing wise in the series, all able to be packaged up into an item – it has been suggested that whilst “old” the SL is well-placed to be folded into a procedural animation system.
Improved animation formats and easier means of animation import into Second Life.
The ability to dynamically set animation priorities for more fluid animation integration (e.g. when you are holding and pointing a gun, you continue to point it as you walk, rather than the avatar’s arm dropping to a walking animation when moving).
Viewer-side animation editor.
Better support for inverse kinematics.
Collaboration between the Puppetry team, the glTF team and any animation project to ensure consistency of decision-making about formats, proper LSL support / calls, etc.
This discussion covered a lot of ground, including the potential for the implementation of an “animation 2.0” system which could potentially operate alongside the existing system (much like PBR materials and “legacy” materials); the benefits in greater adherence to emerging standards – particularly in the area of avatar / skeleton formats and capabilities, and the fact that SL is both well-placed to be a part of defining those standards whilst also being hamper by the fact the existing SL avatar format is a niche product / approach, and more. However, the two key points of the discussion might be summarised as:
Changes to the animation / avatar systems are not projects the Lab is working on at present.
However, the it demonstrates that, as with recent projects, the Lab is looking seriously at enhancing SL and moving it towards more readily understood standards. As such, it is taking the time to ascertain options that are exciting to creators an users and which might be seen s benefiting the platform and its future growth, and so might be formalised into active projects – and include user engagement where appropriate in their development.
In terms of what might be attempted by way of “small-scale” improvements to the animation system, the viewpoint from LL is that the ability to dynamically set animation priorities and adding scaling support to the animation format are seen as providing users / creators with recognisable benefits.
Cover Second Life front-end web properties (Marketplace, secondlife.com, the sign-up pages, the Lab’s corporate pages, etc.).
They are not intended for the discussion of Governance issues, land fees / issues, content creation & tools, viewer or simulator development / projects. Please refer to the SL calendar for information on available meetings for these topics.
A video of the meeting, courtesy of Pantera, can be found embedded at the end of this article (my thanks to her as always!), and subject timestamps to the relevant points in the video are provided. Again, the following is a summary of key topics / discussions, not a full transcript of everything mentioned.
Following recent delays, it had been hoped the MP search updates would be deployed before the end of 2022. Unfortunately, issues with the updates continued to prevent this.
The hope is now to have thing deployed as the “first thing” for web-related deployments, hopefully by the end of January 2023.
As a reminder: this work is essentially the same kind of back-end update as carried out with Web search a few months ago, with the emphasis on faster, more robust performance, together with new search options, including:
Merchant and store names will no long be searched in product searches.
Wildcard (e.g. using *) will be possible.
Better exact matching against search terms
The back-end supports fuzzy matching to better handle typos when inputting searches.
There should be a noticeable increase in speed of search results being returned.
Once running, these updates will allow LL to add-in the relevance engine AI to the Marketplace search (as a separate API entity to the relevance engine already running on the web search).
Summary: A complete re-write of every route by which users can obtain and hold land, from Premium (+Plus) Linden Homes, obtaining Mainland (incl. Abandoned Land), and private island regions, and renting from private estates.
The first user-facing element of this work – the new Land Portal, a central hub from which to get to all aspects of land “ownership” – will likely be deployed by the end of January 2023.
The Portal layout is liable to be a template / proof of concept for overhauling the rest of the Second Life web properties to give them a coherent appearance, make them easier to maintain and add new pages, make SL’s web presence more performant overall and ensure it works on mobile devices as well as desktops / laptops.
Once the Portal has been deployed, the focus will shift to the Linden Homes selection pages.
Work had been expected to resume on Styles (allowing multiple colours, etc., for an item to appear within a single listing rather than each requiring its own listing) following the deployment of the MP Search updates, with the plan for a deployment early-ish in 2023.
As the Search updates deployment has been delayed, the work on Styles is currently being “re-prioritised”.
LL provide an individual RSS feed to all Marketplace stores for announcing new releases, etc.
Whilst not refused as a suggestion, the point was made that implementing and maintaining such a service would not necessarily be easy or direct – as every MP store would require a feed, regardless of whether or not the store owner uses it / people actually subscribe to it via their RSS readers.
There is already functionality available for this within the MP via Favourite Stores > Age: Newest First. This gives a thumbnail view of the most recent 4 releases for every store a user has selected as a Favourite, providing something of an at-a-glance view. It is not as proactive as an RSS feed but has the benefit of already being available.
A request to include a Date Uploaded field in Marketplace descriptions. This would:
Help determine the overall age of a product, allowing it to be better (but not perfectly) assessed against more recent improvements to the platform.
If also included in the Favourite Stores summary list, would further help users determine new releases.
This is seen as being a part of LL’s wish list for MP improvements.
With the move towards supporting the glTF 2.0 specification (initially for PBR materials, but potentially to be expanded to mesh content in general (and eventually possibly animations), these request was made to enable a glTF specific description / search indicator to be added to MP listings.
This is something LL has not considered – but given the shift in emphasis involved in glTF adoption, it is something that should be considered over and above the simple “Mesh” definitions in listings.
For those unaware, details of the glTF / PBR project work can be found in by CCUG meeting summaries.
Requests were also made for:
The ability to preview the contents items sold on the MP as boxes – the problem here being the box is seen as just that; a single object, so a means to “access” its contents and then generate a listing them would need to be built.
The inclusions of triangle counts / VRAM usage / complexity values for objects and attachments – the problems here are numerous, from how would lists be generated / populated (e.g. some form of automated analysis of folder / box contents would be required for accuracy),through how should it be presented without “overloading” listings with information, to the question of whether the results would be properly understood by potential purchasers).
The ability to add multiple items to a shopping basket, but then select which are to be purchased immediately, and which can be “shelved” for a later purchase (if required), rather than having to purchase everything in the basket.
This was also indicated as being on LL’s own wish list of future options for the MP.
Currently, there is no clearly-defined use case for Place Pages, although they do have a number of interesting features and capabilities.
The think at the Lab at the moment is whether Place Pages:
Should be given a degree of work to overhaul them in order to make them more broadly usable, or
Scrap them entirely and replace them with something more attractive to users overall – whatever that might be.
A perceived attraction with Place Pages is the ability for people to show off / promote their Second Life without necessarily having to have others log-in to the platform – or as a means to encourage them to do so – using tools that might be better attuned to presenting SL through the web compared to “ordinary” website building tools
The major reason for not overhauling Place Pages thus far is the fact that they underpin the land auction system.
It is acknowledged that given the plans for overhauling the SL web properties in general, there is potential the produce a version of Place Pages with better integration into SL’s web presence, etc.
User suggestions for Place Pages are welcome, again via the Second Life Jira.
[Video: 27:29-28:02] Communities: a request has been made to expand the range of Second Life communities represented through the Communities landing page (such as the furry community, the sci-fi community, etc). Feature Requests are suggested on these, although it was hinted the number of communities to be rotated on the landing page may be increased.
[Video: 34:24-39:00] “Premium a-la Carte”: whilst indicated as something LL would like to do, the idea of an “a-la carte” set of Premium benefits users can choose from and they pay a fee based on their selection, there are significant issues around general maintenance and billing and tracking which options each user has subscribed to.
What happens if a user selects option A as a part of their package, but then decides they don’t need it, and they want to remove it (and its cost) or swap it for one or more other options + their costs? How frequently should this be allowed? How are the changes to track and recognised by the viewer and relevant back-end services? How is billing to be managed?
How much work is involved in managing an options list (adding new options to it or remove unpopular items from it, in order to maintain its appeal with users, etc.?
How complex any a-la carte system might be was demonstrated by the number of different views expressed in the meeting as to what options might be attractive to different people.
There is also the question as to whether an “a-la carte” option is required, or whether the existing Premium / Premium Plus subscription levels need to be adjusted.
Next Meeting
Wednesday, February 1st, 2023. Venue and time per top of this summary.
The following notes were taken from the Tuesday, January 3rd, 2023 Simulator User Group (SUG) meeting. They form a summary of the items discussed and is not intended to be a full transcript. A video of the entire meeting is embedded at the end of the article for those wishing to review the meeting in full – my thanks to Pantera for recording it.
Server Deployments
On Tuesday, January 3rd, 2023, the simhosts on the Main SLS channel were restarted with no deployment, leaving them on simulator version 576542
On Wednesday, January 4th, 2023, the simhosts on the RC channels should be restarted without any deployment or change to the current release.
Early 2023 Simulator Updates
AS per the previous SUG meeting:
I believe the RC deployment from week #50 (with 30-second sound loop support) was rolled back. If so, this is liable to be one of the first simulator updates for 2023 (targeted for January 11th, 2023), and / or possibly an update comprising a number of HTTP updates, including the accounting for custom HTTP headers (total space for headers will increase to 4k and the limit on the number of headers will be dropped).
Also early 2023 LSL will be updated with new cryptographic signing utilities: llHMAC (per BUG-233005) and llSignRSA and llVerifyRSA (per BUG-233009). These should be useful for script to script and script to external web service communications. These may be included in the above release.
BUG-226463 “llRequestSecureURL() uses self-signed cert” is also “on the radar” but no target date for potential delivery / deployment.
Further (unspecified) improvements will be coming to the Linkset Data (LSD) capabilities.
Available Official Viewers
This list reflects the current status of available official viewers at the start of 2023:
Release viewer: Maintenance P (Preferences, Position and Paste) RC viewer version 6.6.8.576863 Monday, December 12, 2022.
Maintenance (Q)uality RC viewer, version 6.6.9.577220, December 16, 2022.
Performance Floater / Auto-FPS RC viewer, version 6.6.8.576737, November 28, 2022.
VS 2022 RC viewer, version 6.6.8.576310, issued November 4 – utilises Visual Studio 2022 in the Windows build tool chain.
Project viewers:
PBR Materials project viewer, version 7.0.0.577157, December 14, 2022.
This viewer will only function on the following Aditi (beta grid) regions: Materials1; Materials Adult and Rumpus Room 1 through 4.
Puppetry project viewer, version 6.6.8.576972, December 8, 2022.
Love Me Render (LMR) 6 graphics improvements project viewer 6.6.2.573263, July 21, 2022.
In Brief
2023 – General Goals Set by the Lab
In their Review of the Year, Linden Lab outlined the following goals to be delivered during the year (nor all of which are the responsibility of the server / simulator engineering team):
Further performance improvements to increase the quality and visual fidelity of the Second Life experience through the viewer.
New LSL functions that improve the quality of life for scripters.
Avatar customisation improvements to simplify creating your virtual identity, including inventory thumbnails to give a preview of what individual items are.
New user experience upgrades to better fuel retention, engagement, and growth across key parts of the new user journey.
New centralised “hubs” to better connect residents to the communities that match their passions and interests.
First peek at a world and avatar centred mobile-first Second Life experience.
In addition, Rider Linden is an LSL call to allow the replacement of sub-strings in a string call (e.g. non-RegEx behaviour). If implemented, this might most likely be called either llStringReplace() or llReplaceSubString(). As he was still on vacation at this meeting, it is something that will likely be picked up at SUG meetings going forward.
WIBNIs from Users
The following were suggested as a series of “wouldn’t it be nice if LL could provide” requests. Note these were only suggestions / discussion points not projects / ideas the Lab will necessarily implement in the short or long term:
The perennial request for a further focus on region crossing improvements.
A function which could be applied to Animesh objects to allow them to use the head/eye tracking that avatars use when focused on a target (so, for example, scripted scripted Animesh NPCs would “look at” avatars addressing them).
Suggested animation updates:
A procedural animation editor to allow creators / users to set the rules of how avatars walk, run, jump; their timings, how animations play priority wise and mixing wise in the series, all able to be packaged up into an item (a further suggestion for the in-development Puppetry project).
Improved animation formats and easier means of animation import into Second Life.
The ability to dynamically set animation priorities for more fluid animation integration (e.g. when you are holding and pointing a gun, you continue to point it as you walk, rather than the avatar’s arm dropping to a walking animation when moving).
All of these are liable to be topics for further discussion during Puppetry project meetings, but initiated a general discussion on animations and animating avatars which took up the majority of this SUG meeting – please refer to the meeting video below.
The following notes were taken from the Tuesday, December 20th, 2022 Simulator User Group (SUG) meeting. They form a summary of the items discussed and is not intended to be a full transcript. A video of the entire meeting is embedded at the end of the article for those wishing to review the meeting in full – my thanks to Pantera for recording it.
Note: this was a “Winter solstice Party” event, so not a lot to report. The core of any discussions in in the first hour of the video embedded below.
Server Deployments
On Tuesday, December 20th, the simhosts on the Main SLS channel were restarted with no deployment, leaving them on simulator version 576542
On Wednesday, December 21st, the simhosts on the RC channels should be restarted without any deployment or change to the current release.
Early 2023 Simulator Updates
AS per the previous SUG meeting:
I believe the RC deployment from week #50 (with 3-second sound loop support) was rolled back. If so, this is liable to be one of the first simulator updates for 2023 (targeted for January 11th, 2023), and / or possibly an update comprising a number of HTTP updates, including the accounting for custom HTTP headers (total space for headers will increase to 4k and the limit on the number of headers will be dropped).
Also early 2023 LSL will be updated with new cryptographic signing utilities: llHMAC (per BUG-233005) and llSignRSA and llVerifyRSA (per BUG-233009). These should be useful for script to script and script to external web service communications. These may be included in the above release.
BUG-226463 “llRequestSecureURL() uses self-signed cert” is also “on the radar” but no target date for potential delivery / deployment.
Further (unspecified) improvements will be coming to the Linkset Data (LSD) capabilities.
Available Official Viewers
This list reflects the current status of available official viewers at the start of the week:
Release viewer: Maintenance P (Preferences, Position and Paste) RC viewer version 6.6.8.576863 Monday, December 12.
Release channel cohorts:
Maintenance (Q)uality RC viewer, version 6.6.9.577220, December 16.
Performance Floater / Auto-FPS RC viewer, version 6.6.8.576737, November 28.
VS 2022 RC viewer, version 6.6.8.576310, issued November 4 – utilises Visual Studio 2022 in the Windows build tool chain.
Project viewers:
PBR Materials project viewer, version 7.0.0.577157, December 14.
This viewer will only function on the following Aditi (beta grid) regions: Materials1; Materials Adult and Rumpus Room 1 through 4.
Puppetry project viewer, version 6.6.8.576972, December 8.
Love Me Render (LMR) 6 graphics improvements project viewer 6.6.2.573263, July 21.
In Brief
Refer to the video below for more information on the following:
BUG-227303 “collisions makes a script stop running and revert its mono status” has been filed and lead to further discussion on region crossings.
LL is mulling an LSL call to allow the replacement of sub-strings in a string call (e.g. non-RegEx behaviour). If implemented, this might most likely be called either llStringReplace() or llReplaceSubString().
The above lead to a more general discussion on LSL and LSL extensions / transpilers (helping to make LSD more accessible), etc.
Power up for Charge, October 2022 – blog postThe following notes were taken from:
My audio recording and chat log transcript of the Content Creation User Group (CCUG) meeting held on Thursday, December 15th 2022 at 13:00 SLT.
My chat log transcript and video recording by Pantera Północy of the Third-Party Developer Meeting (TPVD) held on Friday, December 16th, 2022 at 13:00 SLT. My thanks to her for the video (embedded towards the end of this article).
These meetings are chaired by Vir Linden, and their dates and times can be obtained from the SL Public Calendar; also note that the following is a summary of the key topics discussed in the meetings and is not intended to be a full transcript of either meeting.
Performance Floater / Auto-FPS RC viewer, version 6.6.8.576737, November 28.
VS 2022 RC viewer, version 6.6.8.576310, issued November 4 – utilises Visual Studio 2022 in the Windows build tool chain.
Project viewers:
Puppetry project viewer, version 6.6.8.576972, December 8.
PBR Materials project viewer, version 7.0.0.576966, December 3.
This viewer will only function on the following Aditi (beta grid) regions: Materials1; Materials Adult and Rumpus Room 1 through 4.
CCUG Specific
glTF Materials and Reflection Probes
Project Summary
To provide support for PBR materials using the core glTF 2.0 specification Section 3.9 and using mikkTSpace tangents, including the ability to have PBR Materials assets which can be applied to surfaces and also traded / sold.
To provide support for reflection probes and cubemap reflections.
The overall goal is to provide as much support for the glTF 2.0 specification as possible.
The project viewer is available via the Alternate Viewers page, but will only work on the following regions on Aditi (the Beta grid): Materials1; Materials Adult and Rumpus Room 1 through 4.
The focus remains on bug and regression issue fixing within the viewer and quality of life improvements.
There has been a discussion on revising the workflow for setting reflection probes & in providing a debug setting to make things easier to see when manipulating reflection probes.
This has been prompted be those testing the PBR viewer setting-up their own reflection probes incorrectly (e.g. by making the probe sphere / cube shiny prior to converting it to a reflection probe – which should NOT be done), failing to achieved the anticipated results and then filing bug reports.
It is possible that some of the required workflow could be factored into the UI itself to off a more intuitive sense as to how reflection probes are supposed to work.
An alternative to this might be to introduce a new “reflection probe” prim type, which includes all the core parameters – however the additional work and messaging required to achieve this would lead to a further lengthening of the project’s development time.
Tutorials to help people get to grips with reflection probes (and, I presume, PBR as a whole are “in the works”).
Screen Space Reflections (SSR): Geenz Linden is working on this within the PBR viewer. Whilst a checkbox for SSR has been added to the viewer, there is further work to be done on the integration and rendering sides (e.g. getting SSR to work on water & transparent surfaces).
Hardware profiling / optimising the viewer is still on-going (although it is likely that whatever is done, SSR will result in a noticeable performance hit if enabled).
[TPV Video: 9:11-10:44] The decision has yet to be made on whether or not to axe the forward rendering (i.e. non-ALM) path from the viewer.
This decision is awaiting more information on hardware performance on lower-spec systems using the updated deferred rendering (i.e. ALM) rendering path.
It is known that Mac rendering performance on the PBR viewer is particularly bad, which may be down to a configuration issue.
In Brief
Mirrors (in some capacity) are described as the “very next thing” the graphics team will commence work on after the PBR / reflection probes work gets to RC status (as Vir Linden pointed out, Mirrors currently require more reflection on how they are to be implemented!).
The meeting in places broadened into general discussion on content breakage (and the need, where possible, to maintain the functionality of content users have purchased in expectation of its longevity; the benefits / disadvantages with morph bones versus blend shapes for customising the avatar shape (and the possible routes to an “avatar 2.0”), the need for a better animation system (or even an actual animation system), etc.
Much of this was more esoteric in nature at this point in time, although the likes of Puppetry is laying the foundations for broader animation work; as does the glTF 2.0 specification, which is the baseline specification for the PBR / reflections probe work, and will be expanded upon in future projects – such as with mesh uploads.
In terms of animation systems, some of the groundwork is already in place inasmuch as SL already effectively treats morph bones and animated bones as individual animation tracks, offering to potential for this to ne “unwrapped” and moved to a glTF approach to animation management.
Some of the above conversation touched on the ideas of market renewal / fragmentation. For example: the introduction of reflection probes offers a degree of market renewal for creators of buildings, skyboxes, etc., through the provisioning of new builds (or new versions of existing builds) leveraging the capability; however, the adoption of an upload schema for fully customised avatar skeletons might lead to greater market fragmentation (how do you ensure, for example, that human animation/pose A will work equally on custom human (style) avatar skeletons W, X, Y and Z, or will they each require custom animations?).
A new project (commencing in 2023) to add thumbnail previews to inventory, allowing users to see a small image of a given object within inventory (thumbnails can be individual items or, if preferred an entire folder).
The first phase of the work is determining how to generate the thumbnail images in the first place on a manual or (preferably) automated basis, and then ensure it maintains an association with the object to which it is related (e.g. so if an item is sold or transferred to another user, the thumbnail goes with it).
Once this has been decided, the next phase will be to build-out the UI so that such thumbnails can be viewed from inventory
It is likely that thumbnails will have a fixed / limited size resolution, and will not be subject to any upload fee.
This work:
Is not an adjunct (or related) to the Outfit previews currently available in the viewer.
Will not prevent creators from including high resolution images with their products if they wish.
Will only apply to inventory objects where it makes sense (e.g. textures and notecards would likely be excluded).
Is seen as a possible foundational piece to adding new fields to the inventory database, which could open the door to further information fields being added to inventory in the future,
Following the switch-over to using Github for viewer code repositories on Monday, November 21st, 2022, work is now progressing on “Phase 2”.
This is remaining TeamCity operations with Github Actions for viewer builds (so those pulling an official viewer repo will be able to build it directly).
This work will also incorporate the rebuilding of those third-party libraries involved in the viewer build process, as and where required.
In Brief
[Video 16:10-18:41] LL has been digging into viewer crash rates by operating system, with the note that the number of crashes on the Mac OS appears to be disproportionately high. It is hoped that if the underlying causes can be readily identified, these issues can be subjected to rapid fixing. It was not clear at the meeting if those TPVs supporting OS X are seeing a similar elevated crash rate.
[Video 20:40-21:35] It is estimated that the split between viewer operating systems for the official viewer is roughly:
1-2% Linux (although this does not include running the Windows viewer under emulation on Linux).
6% Mac.
The rest: Windows.
Firestorm appears to mirror the above.
[Video: 35:08-36:10] It is possible the the work on the viewer library refresh will allow LL to look again at the issue of providing a Linux viewer build.
The meeting has a general discussion on operating systems, & variants (64-bit vs. 32-bit), etc, and the lack of an official Linux build,
A reminder that Microsoft ceases official support for Windows 8 on January 10th, 2023. This means that from that date, Windows 8 will no longer be officially supported by Second Life as a viewer operating system, and LL will not guarantee the viewer will run as expected on Win 8 going forward from that date.
Linden Lab Holiday Closure
A reminder that Linden Lab will be effectively closed (outside of Support cover) from end of business (PST) on Friday, December 23rd, 2022 through until start of business (PST) on Monday, January 2nd, 2023.
Puppetry demonstration via Linden Lab – see below. Demos video with the LL comment “We have some basic things working with a webcam and Second Life but there’s more to do before it’s as animated as we want.”
The following notes have been taken from chat logs and audio recording of the Thursday, December 8th Puppetry Project meetings held at the Castelet Puppetry Theatre on Aditi. These meetings are generally held on alternate weeks to the Content Creation User Group (CCUG), on same day / time (Thursdays at 13:00 SLT).
Notes in these summaries are not intended to be a full transcript of every meeting, but to highlight project progress / major topics of discussion.
Project Summary
Previously referred to as “avatar expressiveness”, Puppetry is intended to provide a means by which avatars can mimic physical world actions by their owners (e.g. head, hand, arm movements) through tools such as a webcam and using technologies like inverse kinematics (IK) and the LLSD Event API Plug-in (LEAP) system.
Note that facial expressions and finger movements are not currently enabled.
Most movement is in the 2D plain (e.g., hand movements from side-to-side but not forward / back), due to limitations with things like depth of field tracking through a webcam, which has yet to be addressed.
The back-end support for the capability is only available on Aditi (the Beta grid) and within the following regions: Bunraku, Marionette, and Castelet.
Puppetry requires the use of a dedicated viewer, the Project Puppetry viewer, available through the official Second Life Alternate Viewers page.
No other special needs beyond the project viewer are required to “see” Puppetry animations. However, to use the capability to animate your own avatar and broadcast the results, requires additional work – refer to the links below.
There is now a Puppetry Discord channel – those wishing to join it should contact members of LL’s puppetry team, e.g. Aura Linden, Simon Linden, Rider Linden, Leviathan Linden (not a full list of names at this time – my apologies to those involved whom I have missed).
Bugs, Feature Requests and Code Submissions
For those experimenting with Puppetry, Jiras (bug reports / fixes or feature requests) should be filed with “[Puppetry]” at the start of the Jira title.
A new version of the Puppetry project viewer – version 6.6.8.576972 was issued on December 8th, available on the Alternate Viewers page (until a further version is issued).
This version includes an overhaul to the protocol used between LEAP plug-ins and the viewer. For example, Inverse Kinematics calculations are done earlier in the process which will make viewer performance better when more than one avatar is using Puppetry.
Please be certain to use both a new viewer and a new set of plug-ins from https://github.com/secondlife-3p, and update any projects or code you might be working on.
This version may have a crash bug.
Leviathan Linden is still working on updating the wiki documentation to reflect the new API.
Leviathan Linden demonstrating the use of puppetry to move his avatar whilst doing the “pane of glass” mime in front of a suitable capture device. Note that his legs remain static (moving in line with his hips) as puppetry does not (yet) support full body tracking
We changed the way Puppetry expects to get its data for two reasons: 1) we want to only do IK for your own avatar, then just send the joint rotations to everybody else; 2) if someone writes a plug-in that happens to know best what all the joint rotations should be (e.g. it has done its own IK, or is doing full mocap) then it can just specify all parent-frame rotations of the joints. So, now that THAT plug-in mode is unblocked, we can start trying to fix our own IK.
– Leviathan Linden explaining the changes to the way puppetry and managed by the viewer
In Brief
There are currently no updates to the Inverse Kinematics (IK); it is described as being “hard”.
It has been suggested that the viewer could “do more” in respects of IK, etc.
However, this information needs to be transmitted to other viewers able to “see” what is going on, which requires messaging and updates through the simulator, which can lead to the viewer being less than trustworthy in terms of what it is showing and what is going on (due to missed updates, etc due to the bandwidth load).
But, if the simulator is tasked with managing all the computations for IK and sending the results to connected viewers (reducing the amount of traffic and potential loss of messaging), it puts a potentially high compute load on the simulator (imagine the simulator trying to manage the IK for 50+ avatars at an event, tracking the movement, interactions, etc.).
A potential trade off here is to have a viewer run the IK calculations for the avatar it is controlling, and package that information for streaming to other viewers connected to the region’s simulator with minimal sanity checking (e.g. to ensure the avatar’s position in the viewer is properly constrained to within a few metres of its location as calculated by the simulator). On receipt, the sanity-checked data can then be played back without the need for the receiving viewer having to carry out IK calculations for itself on the avatar(s) it is “watching”, and carrying out some minimal sanity checks of its own.
Rider Linden is investigating having the simulator track the motion from a puppetry viewer in a way that does not impact simulator performance “too badly”. The options he’s looking at are:
Having the simulator suck the data out of the puppetry messages as they are sent through it, or
Using a new message the viewer can use to report the locations of it’s attachment points, and the simulator tracks these – which Rider sees as the preferred option.
This later method could – among other things – be expanded to work with animations in general. In addition, if tied tied in to the entire animation system (viewer computes its animation frame puppetry+legacy) to produce results, then the results could be made to scripts.
In regards to interfaces for this, Rider is of the opinion that scripts should reference attachment points since those can act as proxies for bone locations and most general creators, scripters and residents are familiar with them. and it avoids having to introduce a new concept of bones into LSL.
Collisions: the above spawned a related discussion on providing additional data such as geometry information (e.g. spherical bounding radius or a shape approximation) to allow collisions to be enabled from IK, and allowing “snap to” functionality (e.g. you reach for a glass and the avatar hand snaps to it on detecting the collision).
However, rather than allowing allow physics collisions on an attachment points (which might over-complicate the avatar model in the Havok physics engine), Rider suggested having a property (sphere) that could be set on an attachment that enabled it as a physics volume.
My thought is that collision aware attachment points, along with being able to detect and set their positions in space will be enough to get us 90% of the way towards being able to hold hands in world.
– Rider Linden
Leviathan noted there was a bug where an avatar with non-unity scale on its bones would be broken under puppetry (misalignment of bones). This should now be fixed, but it allowed him to add the theoretical ability to modify the scale of a joint in its parent-frame (an example plug-in script to demonstrate ho this works has yet to be written).
