Where Our Journey Begins, November 2022 – blog post
The following notes were taken from
My audio recording and chat log transcript of the Content Creation User Group (CCUG) meeting held on Thursday, January 19th 2023 at 13:00 SLT.
Pantera’s video of the Third Part Viewer Developer (TPVD) meeting held on Friday, January 20th, 2023 at 13:00 SLT, embedded at the end of this article.
These meetings are chaired by Vir Linden, and their dates and times can be obtained from the SL Public Calendar; also note that the following is a summary of the key topics discussed in the meetings and is not intended to be a full transcript of all points raised.
Note, The TPVD meeting was abbreviated to 20 mins.
Official Viewers Status – TPV Meeting
Available Viewers
On Thursday, January 19th, 2023, the Maintenance R RC viewer was updated to version 6.6.9.577678 – translation updates and the return of slam bits.
On Wednesday, January18th, 2023:
The Maintenance (Q)uality RC viewer was updated to version on 6.6.9.577581 – new Debug settings UI, quality of life improvements.
The PBR Materials project viewer updated to version 7.0.0.577610, on January 19, 2023 – SSR support.
This viewer has been reported as “broken” when running on systems with AMD GPUs, a situation that was being investigation as these notes were being written.
This leaves the rest of the currently-available official viewer as:
Release viewer: Maintenance P (Preferences, Position and Paste) RC viewer version 6.6.8.576863 Monday, December 12, 2022.
Release Candidate viewers:
Performance Floater / Auto-FPS RC viewer, version 6.6.9.577251, January 4, 2023.
Project viewers:
Puppetry project viewer, version 6.6.8.576972, December 8, 2022.
General Viewer Notes
Vir pointed out that as Microsoft has ended support for Windows 8 on January 10th, 2023, it is no longer regarded as a supported operating system for running Second Life, and the viewer will not be tested against it (and the System Requirements page has been updated to reflect this, specifying Windows 10+ most recent service pack as the baseline supported Windows OS.
The second phase of the Github work in on-going, notably updating all the viewer build libraries.
Inventory Enhancement Project – Both Meetings
Linden Lab is looking to enhance the Inventory system.
The first element of this work is to be the addition of a fixed-resolution thumbnail preview capability, allowing users to see a small image of a given object (where this makes sense – so the likes of note cards and scripts would be excluded) within inventory, with these thumbnails either being of individual items or entire folders.
This work has now started, but it will be “some time” before there is anything user-facing to show.
A code contribution from Kitty Barnett (Catznip) for an inventory texture tool tip / preview may well be folded in to this work.
Once the thumbnail preview work has been completed, it is possible the Lab will look to further enhancements to inventory management. One future enhancement under consideration is support for folders to be included in the Contents inventory of individual objects.
glTF Materials and Reflection Probes – CCUG
Project Summary
To provide support for PBR materials using the core glTF 2.0 specification Section 3.9 and using mikkTSpace tangents, including the ability to have PBR Materials assets which can be applied to surfaces and also traded / sold.
To provide support for reflection probes and cubemap reflections.
The overall goal is to provide as much support for the glTF 2.0 specification as possible.
The project viewer is available via the Alternate Viewers page, but will only work on the following regions on Aditi (the Beta grid): Materials1; Materials Adult and Rumpus Room 1 through 4.
Screen Space Reflections (SSR) has now been integrated into the Project Viewer, (January 19th onwards), although work around this is still being finalised, notably mixed Reflection Probes with SSR.
Some fixing is required to the default colour curves in the viewer.
Stability issues have been noted on Intel integrated graphics hardware, and these are being investigated / fixed.
There have now been a total of two further weeks focused on optimising the viewer’s performance, and the results of this work should be available in the next update to the Project Viewer.
UI updates: the viewer will likely see some UI updates as it progresses:
The next update will likely include a new dedicated icon for on the Build floater for creating Reflection Probes.
The drop-down in the Build floater Textures tab (Currently Materials and Media on the release viewer, may be re-labelled for PBR (where it is Materials, Media, PBR) to more clearly differentiate between the current materials maps system and “new”” PBR materials.
Similarly, the new Materials Inventory folder in the Project Viewer may also be renamed to avoid the potential for confusion between the use of PBR materials assets and the current materials maps system.
Further, the menu option Build → Upload → Materials (in the Project Viewer) may be reamed to more clearly reflect it is for PBR materials, not material maps.
Exactly was terms will be used is TBD, but the general push (from LL’s perspective is to use meaningful lbeels on options and menus which can be Googled with some relevance by those wanting to know more about the underpinning file formats / standards, etc.
Work is going into messaging between the back-end and the viewer to ensure it will scale once the viewer starts to see widespread use.
In Brief
As the PBR Materials project is approaching a possible RC release, the questions has been asked as to what is next on the graphics front. While no formal decision has been made, options include:
A return to looking at adding something like Vulkan / MoltenVK (for OS X) API support alongside of the OpenGL API (which has been slated for deprecation by Apple and is growing increasingly long in the tooth). This work is being looked at more as a performance optimisation rather than a visual boost to rendering (e.g. allowing SL to be less CPU-bound and make more use of the GPU, reducing the volume of draw calls, etc.).
Further expansion of the glTF work to include mesh (and moving away from Collada .DAE), animations, etc. But again, no definitive plans / direction has been agreed within the Lab.
While there is a appetite at the Lab to support as much as the glTF 2.0 specification, some aspects will be excluded from the list – mentioned at the meeting were Setting the filter mode and sampler state of textures, both of which were described as “problematic” for implementation within SL.
Overhauling / updating support / options for ground textures.
The CCUG meeting saw a general discussion on animations, the skeleton and all that it involves (bones, attachment points, collision volumes, rigging, etc., and the rules / policies surrounding them, couched initially in the potential for expanding the skeleton / avatar system – for which there are currently no plans to do so beyond what has already been done.
Pivot points: (related to the above) past CCUG meeting saw some extended discussion on implementing pivot points on the skeleton, and LL indicated they would investigate this. Since then, it has become increasingly apparent that a node hierarchy would be beneficial and potentially easier to implement; ergo pivot points have been pulled into the pot for a future hierarchy project.
There is a known bug where Premium Plus users – who gain free texture uploads – are being charged for textures included in mesh uploads. An upcoming simulator-side maintenance update should correct this.
A request was made at the CCUG meeting to allow mesh uploads greater than the 64m single object size limit, to make it easier to import large structures / scenes without having to curt them into sections (and thus avoid potential issues in fit, scale, update, etc, and can result in Interest List related rendering issues).
LL is unlikely to change the limit, but would prefer to provide improved tools that can help content creators / scene builders.
One should tool discussed – which is currently not in development – is that of a “scene view” tool which identifies every individual instance of an object in scene, and provides a mechanism to allow (subject to permissions, obviously) them to be swapped-out to fix things like “broken” items / those for which there is an update / replacement.
Puppetry demonstration via Linden Lab – see below. Demos video with the LL comment “We have some basic things working with a webcam and Second Life but there’s more to do before it’s as animated as we want.”
The following notes have been taken from chat logs and audio recording of the Thursday, January 12th, 2023 Puppetry Project meetings held at the Castelet Puppetry Theatre on Aditi. These meetings are generally held on alternate weeks to the Content Creation User Group (CCUG), on same day / time (Thursdays at 13:00 SLT).
Notes in these summaries are not intended to be a full transcript of every meeting, but to highlight project progress / major topics of discussion.
Project Summary
General description of the project and its inception:
LL’s renewed interest in puppetry was primarily instigated by Philip joining LL as official advisor, and so it really was about streaming mocap. That is what Philip was interested in and why we started looking at it again. However since Puppetry’s announcement what I’ve been hearing from many SL Residents is: what they really want from “puppetry” is more physicality of the avatar in-world: picking up objects, holding hands, higher fidelity collisions.
As a result, that is what I’ve been contemplating: how to improve the control and physicality of the the avatar. Can that be the new improved direction of the Puppetry project? How to do it?
Leviathan Linden
Previously referred to as “avatar expressiveness”, Puppetry is intended to provide a means by which avatars can mimic physical world actions by their owners (e.g. head, hand, arm movements) through tools such as a webcam and using technologies like inverse kinematics (IK) and the LLSD Event API Plug-in (LEAP) system.
Note that facial expressions and finger movements are not currently enabled.
Most movement is in the 2D plain (e.g., hand movements from side-to-side but not forward / back), due to limitations with things like depth of field tracking through a webcam, which has yet to be addressed.
The back-end support for the capability is only available on Aditi (the Beta grid) and within the following regions: Bunraku, Marionette, and Castelet.
Puppetry requires the use of a dedicated viewer, the Project Puppetry viewer, available through the official Second Life Alternate Viewers page.
No other special needs beyond the project viewer are required to “see” Puppetry animations. However, to use the capability to animate your own avatar and broadcast the results, requires additional work – refer to the links below.
This project is taking in a lot of additional ideas – animation standards, improving the current animation system, enabling truer avatar / avatar and avatar object interactions such that it is likely to evolve into a rolling development, with immediate targets for development / implementation as they are agreed upon, to be followed by future enhancements.
As such, much of what goes into the meetings at present is general discussion and recommendations for consideration, rather than confirmed lines o development.
There is a Puppetry Discord channel – those wishing to join it should contact members of LL’s puppetry team, e.g. Aura Linden, Simon Linden, Rider Linden, Leviathan Linden (not a full list of names at this time – my apologies to those involved whom I have missed).
Bugs, Feature Requests and Code Submissions
For those experimenting with Puppetry, Jiras (bug reports / fixes or feature requests) should be filed with “[Puppetry]” at the start of the Jira title.
See: OPEN-375: “LSL Functions for reading avatar animation positions”.
Rider Linden is starting to look at LSL integration – the first step being to make the simulator aware of what is actually animating.
Currently, the code he has developed lets the server know the position of an avatars attachment points; this sends details of 55 points (HUD points excepted). Attachment points have been selected over bones, as the simulator already has a solid concept of attachment points, and it avoids complications with rigged meshes “doing their own thing” with bone positions.
A concern with this is the number of updates being sent to the server for processing.
One idea is to refine the the code so that only the attachment points which change relative to the avatar centre (avatar frame/Local Position relative to the avatar) actually send information to the server, in order to reduce the number of updates being generated.
Another idea might be to only send updates every n frames, rather than every frame. This would reduce the fidelity of movement, but could still provide sufficient data while reducing the load on the simulator, particularly where multiple avatars in a region are using puppetry.
This issue is related to synchronising puppetry actions across multiple viewers as well; a long-standing issues, given that animation playback of animations is viewer-side, and not genuinely across viewers (the resync function found in some TPVs only does so locally).
All of the above lead to a discussions of ways and means to best allow LSL integration with animations and ensure a reasonable transmission of results together with decent synchronisation between the viewer and the simulator, whether by frame count or time stamp, in order to ensure predictability of results across multiple viewers. .
In addition, the discussion included the advantage in enhancing Second Life to support procedural animations as well as the current canned animations.
Rider is also looking into a script enhancement to register collisions.
There was some conflating of ideas during the discussion – immediate first steps in opening Puppetry to LSL, and more far reaching goals – setting position, registering collisions (per the above), defining better interpolation for positioning (e.g. as defined in the Khronos glTF specification), etc., which caused a degree of confusion.
However, the openness towards making Puppetry a good foundation for future enhancement (such as moving more to procedural-based animations, enabling SL to support “industry standard” animation workflows to encourage animators into the platform, etc., remains, together with (hopefully) enabling more realistic avatar / avatar and avatar / object interactions.
That said, Simon Linden did offer a not of caution to all discussing the work:
Not to pop the bubble, but every one please keep in mind all the stuff we’re talked about is experimental and really interesting. I have no idea what we can make into real features and what can work with crowds and all the other interesting problems to make it happen well – we’ll see what we all can do this year 🙂
Mullein Woods, November 2022 – blog post
The following notes were taken from m y audio recording and chat log transcript of the Content Creation User Group (CCUG) meeting held on Thursday, January 5th 2023 at 13:00 SLT.
These meetings are chaired by Vir Linden, and their dates and times can be obtained from the SL Public Calendar; also note that the following is a summary of the key topics discussed in the meeting and is not intended to be a full transcript of all points raised.
Official Viewers Status
On Wednesday, January 4th, 2023:
The Maintenance Q(uality) RC viewer updated to version 6.6.9.577418.
The Performance Floater / Auto-FPS RC viewer updated to version 6.6.9.577251.
Both the VS 2022 Build RC viewer and the LMR6 project viewer have been withdrawn.
This leaves the rest of the currently-available official viewer as:
Release viewer: Maintenance P (Preferences, Position and Paste) RC viewer version 6.6.8.576863 Monday, December 12, 2022.
Project viewers:
PBR Materials project viewer, version 7.0.0.577157, December 14, 2022. Note: this viewer will only function on the following Aditi (beta grid) regions: Materials1; Materials Adult and Rumpus Room 1 through 4.
Puppetry project viewer, version 6.6.8.576972, December 8, 2022.
Inventory Enhancement Project
Linden Lab is looking to enhance the Inventory system.
The first element of this work is to be the addition of a fixed-resolution thumbnail preview capability, allowing users to see a small image of a given object (where this makes sense – so the likes of note cards and scripts would be excluded) within inventory, with these thumbnails either being of individual items or entire folders.
The first phase of the work is determining how to generate the thumbnail images and ensure they maintain an association with the objects to which they are related (e.g. so if an item is sold or transferred to another user, the thumbnail goes with it).
Once this has been decided, the next phase will be to build-out the UI so that such thumbnails can be viewed from inventory.
This work will not replace the Outfit Folder image capability nor will it prevent creators from including high resolution images with their products if they wish.
Once the thumbnail preview work has been completed, it is possible the Lab will look to further enhancements to inventory management. One future enhancement under consideration is support for folders to be included in the Contents inventory of individual objects.
glTF Materials and Reflection Probes
Project Summary
To provide support for PBR materials using the core glTF 2.0 specification Section 3.9 and using mikkTSpace tangents, including the ability to have PBR Materials assets which can be applied to surfaces and also traded / sold.
To provide support for reflection probes and cubemap reflections.
The overall goal is to provide as much support for the glTF 2.0 specification as possible.
The project viewer is available via the Alternate Viewers page, but will only work on the following regions on Aditi (the Beta grid): Materials1; Materials Adult and Rumpus Room 1 through 4.
The focus remains on bug and regression issue fixing within the viewer and quality of life improvements, particularly as in wider grid testing, it has been found the PBR viewer can only generate single-digit FPS in some regions.
Screen Space Reflections (SSR): Geenz Linden continues to work on integrating SSR into the PBR viewer, but is encountering issues.
Animation System Enhancements – A Discussion
In response to requests for the animation system to be improved (e.g. via CCUG meetings, as a result of the Puppetry project, etc.), Vir Linden asked those at the meeting to state what they see as the most important changes / updates they would like to see. Responses included those expressed at the Server User Group meeting earlier in the week:
A procedural animation system to allow creators / users to set the rules of how avatars walk, run, jump; their timings, how animations play priority wise and mixing wise in the series, all able to be packaged up into an item – it has been suggested that whilst “old” the SL is well-placed to be folded into a procedural animation system.
Improved animation formats and easier means of animation import into Second Life.
The ability to dynamically set animation priorities for more fluid animation integration (e.g. when you are holding and pointing a gun, you continue to point it as you walk, rather than the avatar’s arm dropping to a walking animation when moving).
Viewer-side animation editor.
Better support for inverse kinematics.
Collaboration between the Puppetry team, the glTF team and any animation project to ensure consistency of decision-making about formats, proper LSL support / calls, etc.
This discussion covered a lot of ground, including the potential for the implementation of an “animation 2.0” system which could potentially operate alongside the existing system (much like PBR materials and “legacy” materials); the benefits in greater adherence to emerging standards – particularly in the area of avatar / skeleton formats and capabilities, and the fact that SL is both well-placed to be a part of defining those standards whilst also being hamper by the fact the existing SL avatar format is a niche product / approach, and more. However, the two key points of the discussion might be summarised as:
Changes to the animation / avatar systems are not projects the Lab is working on at present.
However, the it demonstrates that, as with recent projects, the Lab is looking seriously at enhancing SL and moving it towards more readily understood standards. As such, it is taking the time to ascertain options that are exciting to creators an users and which might be seen s benefiting the platform and its future growth, and so might be formalised into active projects – and include user engagement where appropriate in their development.
In terms of what might be attempted by way of “small-scale” improvements to the animation system, the viewpoint from LL is that the ability to dynamically set animation priorities and adding scaling support to the animation format are seen as providing users / creators with recognisable benefits.
Power up for Charge, October 2022 – blog postThe following notes were taken from:
My audio recording and chat log transcript of the Content Creation User Group (CCUG) meeting held on Thursday, December 15th 2022 at 13:00 SLT.
My chat log transcript and video recording by Pantera Północy of the Third-Party Developer Meeting (TPVD) held on Friday, December 16th, 2022 at 13:00 SLT. My thanks to her for the video (embedded towards the end of this article).
These meetings are chaired by Vir Linden, and their dates and times can be obtained from the SL Public Calendar; also note that the following is a summary of the key topics discussed in the meetings and is not intended to be a full transcript of either meeting.
Performance Floater / Auto-FPS RC viewer, version 6.6.8.576737, November 28.
VS 2022 RC viewer, version 6.6.8.576310, issued November 4 – utilises Visual Studio 2022 in the Windows build tool chain.
Project viewers:
Puppetry project viewer, version 6.6.8.576972, December 8.
PBR Materials project viewer, version 7.0.0.576966, December 3.
This viewer will only function on the following Aditi (beta grid) regions: Materials1; Materials Adult and Rumpus Room 1 through 4.
CCUG Specific
glTF Materials and Reflection Probes
Project Summary
To provide support for PBR materials using the core glTF 2.0 specification Section 3.9 and using mikkTSpace tangents, including the ability to have PBR Materials assets which can be applied to surfaces and also traded / sold.
To provide support for reflection probes and cubemap reflections.
The overall goal is to provide as much support for the glTF 2.0 specification as possible.
The project viewer is available via the Alternate Viewers page, but will only work on the following regions on Aditi (the Beta grid): Materials1; Materials Adult and Rumpus Room 1 through 4.
The focus remains on bug and regression issue fixing within the viewer and quality of life improvements.
There has been a discussion on revising the workflow for setting reflection probes & in providing a debug setting to make things easier to see when manipulating reflection probes.
This has been prompted be those testing the PBR viewer setting-up their own reflection probes incorrectly (e.g. by making the probe sphere / cube shiny prior to converting it to a reflection probe – which should NOT be done), failing to achieved the anticipated results and then filing bug reports.
It is possible that some of the required workflow could be factored into the UI itself to off a more intuitive sense as to how reflection probes are supposed to work.
An alternative to this might be to introduce a new “reflection probe” prim type, which includes all the core parameters – however the additional work and messaging required to achieve this would lead to a further lengthening of the project’s development time.
Tutorials to help people get to grips with reflection probes (and, I presume, PBR as a whole are “in the works”).
Screen Space Reflections (SSR): Geenz Linden is working on this within the PBR viewer. Whilst a checkbox for SSR has been added to the viewer, there is further work to be done on the integration and rendering sides (e.g. getting SSR to work on water & transparent surfaces).
Hardware profiling / optimising the viewer is still on-going (although it is likely that whatever is done, SSR will result in a noticeable performance hit if enabled).
[TPV Video: 9:11-10:44] The decision has yet to be made on whether or not to axe the forward rendering (i.e. non-ALM) path from the viewer.
This decision is awaiting more information on hardware performance on lower-spec systems using the updated deferred rendering (i.e. ALM) rendering path.
It is known that Mac rendering performance on the PBR viewer is particularly bad, which may be down to a configuration issue.
In Brief
Mirrors (in some capacity) are described as the “very next thing” the graphics team will commence work on after the PBR / reflection probes work gets to RC status (as Vir Linden pointed out, Mirrors currently require more reflection on how they are to be implemented!).
The meeting in places broadened into general discussion on content breakage (and the need, where possible, to maintain the functionality of content users have purchased in expectation of its longevity; the benefits / disadvantages with morph bones versus blend shapes for customising the avatar shape (and the possible routes to an “avatar 2.0”), the need for a better animation system (or even an actual animation system), etc.
Much of this was more esoteric in nature at this point in time, although the likes of Puppetry is laying the foundations for broader animation work; as does the glTF 2.0 specification, which is the baseline specification for the PBR / reflections probe work, and will be expanded upon in future projects – such as with mesh uploads.
In terms of animation systems, some of the groundwork is already in place inasmuch as SL already effectively treats morph bones and animated bones as individual animation tracks, offering to potential for this to ne “unwrapped” and moved to a glTF approach to animation management.
Some of the above conversation touched on the ideas of market renewal / fragmentation. For example: the introduction of reflection probes offers a degree of market renewal for creators of buildings, skyboxes, etc., through the provisioning of new builds (or new versions of existing builds) leveraging the capability; however, the adoption of an upload schema for fully customised avatar skeletons might lead to greater market fragmentation (how do you ensure, for example, that human animation/pose A will work equally on custom human (style) avatar skeletons W, X, Y and Z, or will they each require custom animations?).
A new project (commencing in 2023) to add thumbnail previews to inventory, allowing users to see a small image of a given object within inventory (thumbnails can be individual items or, if preferred an entire folder).
The first phase of the work is determining how to generate the thumbnail images in the first place on a manual or (preferably) automated basis, and then ensure it maintains an association with the object to which it is related (e.g. so if an item is sold or transferred to another user, the thumbnail goes with it).
Once this has been decided, the next phase will be to build-out the UI so that such thumbnails can be viewed from inventory
It is likely that thumbnails will have a fixed / limited size resolution, and will not be subject to any upload fee.
This work:
Is not an adjunct (or related) to the Outfit previews currently available in the viewer.
Will not prevent creators from including high resolution images with their products if they wish.
Will only apply to inventory objects where it makes sense (e.g. textures and notecards would likely be excluded).
Is seen as a possible foundational piece to adding new fields to the inventory database, which could open the door to further information fields being added to inventory in the future,
Following the switch-over to using Github for viewer code repositories on Monday, November 21st, 2022, work is now progressing on “Phase 2”.
This is remaining TeamCity operations with Github Actions for viewer builds (so those pulling an official viewer repo will be able to build it directly).
This work will also incorporate the rebuilding of those third-party libraries involved in the viewer build process, as and where required.
In Brief
[Video 16:10-18:41] LL has been digging into viewer crash rates by operating system, with the note that the number of crashes on the Mac OS appears to be disproportionately high. It is hoped that if the underlying causes can be readily identified, these issues can be subjected to rapid fixing. It was not clear at the meeting if those TPVs supporting OS X are seeing a similar elevated crash rate.
[Video 20:40-21:35] It is estimated that the split between viewer operating systems for the official viewer is roughly:
1-2% Linux (although this does not include running the Windows viewer under emulation on Linux).
6% Mac.
The rest: Windows.
Firestorm appears to mirror the above.
[Video: 35:08-36:10] It is possible the the work on the viewer library refresh will allow LL to look again at the issue of providing a Linux viewer build.
The meeting has a general discussion on operating systems, & variants (64-bit vs. 32-bit), etc, and the lack of an official Linux build,
A reminder that Microsoft ceases official support for Windows 8 on January 10th, 2023. This means that from that date, Windows 8 will no longer be officially supported by Second Life as a viewer operating system, and LL will not guarantee the viewer will run as expected on Win 8 going forward from that date.
Linden Lab Holiday Closure
A reminder that Linden Lab will be effectively closed (outside of Support cover) from end of business (PST) on Friday, December 23rd, 2022 through until start of business (PST) on Monday, January 2nd, 2023.
Puppetry demonstration via Linden Lab – see below. Demos video with the LL comment “We have some basic things working with a webcam and Second Life but there’s more to do before it’s as animated as we want.”
The following notes have been taken from chat logs and audio recording of the Thursday, December 8th Puppetry Project meetings held at the Castelet Puppetry Theatre on Aditi. These meetings are generally held on alternate weeks to the Content Creation User Group (CCUG), on same day / time (Thursdays at 13:00 SLT).
Notes in these summaries are not intended to be a full transcript of every meeting, but to highlight project progress / major topics of discussion.
Project Summary
Previously referred to as “avatar expressiveness”, Puppetry is intended to provide a means by which avatars can mimic physical world actions by their owners (e.g. head, hand, arm movements) through tools such as a webcam and using technologies like inverse kinematics (IK) and the LLSD Event API Plug-in (LEAP) system.
Note that facial expressions and finger movements are not currently enabled.
Most movement is in the 2D plain (e.g., hand movements from side-to-side but not forward / back), due to limitations with things like depth of field tracking through a webcam, which has yet to be addressed.
The back-end support for the capability is only available on Aditi (the Beta grid) and within the following regions: Bunraku, Marionette, and Castelet.
Puppetry requires the use of a dedicated viewer, the Project Puppetry viewer, available through the official Second Life Alternate Viewers page.
No other special needs beyond the project viewer are required to “see” Puppetry animations. However, to use the capability to animate your own avatar and broadcast the results, requires additional work – refer to the links below.
There is now a Puppetry Discord channel – those wishing to join it should contact members of LL’s puppetry team, e.g. Aura Linden, Simon Linden, Rider Linden, Leviathan Linden (not a full list of names at this time – my apologies to those involved whom I have missed).
Bugs, Feature Requests and Code Submissions
For those experimenting with Puppetry, Jiras (bug reports / fixes or feature requests) should be filed with “[Puppetry]” at the start of the Jira title.
A new version of the Puppetry project viewer – version 6.6.8.576972 was issued on December 8th, available on the Alternate Viewers page (until a further version is issued).
This version includes an overhaul to the protocol used between LEAP plug-ins and the viewer. For example, Inverse Kinematics calculations are done earlier in the process which will make viewer performance better when more than one avatar is using Puppetry.
Please be certain to use both a new viewer and a new set of plug-ins from https://github.com/secondlife-3p, and update any projects or code you might be working on.
This version may have a crash bug.
Leviathan Linden is still working on updating the wiki documentation to reflect the new API.
Leviathan Linden demonstrating the use of puppetry to move his avatar whilst doing the “pane of glass” mime in front of a suitable capture device. Note that his legs remain static (moving in line with his hips) as puppetry does not (yet) support full body tracking
We changed the way Puppetry expects to get its data for two reasons: 1) we want to only do IK for your own avatar, then just send the joint rotations to everybody else; 2) if someone writes a plug-in that happens to know best what all the joint rotations should be (e.g. it has done its own IK, or is doing full mocap) then it can just specify all parent-frame rotations of the joints. So, now that THAT plug-in mode is unblocked, we can start trying to fix our own IK.
– Leviathan Linden explaining the changes to the way puppetry and managed by the viewer
In Brief
There are currently no updates to the Inverse Kinematics (IK); it is described as being “hard”.
It has been suggested that the viewer could “do more” in respects of IK, etc.
However, this information needs to be transmitted to other viewers able to “see” what is going on, which requires messaging and updates through the simulator, which can lead to the viewer being less than trustworthy in terms of what it is showing and what is going on (due to missed updates, etc due to the bandwidth load).
But, if the simulator is tasked with managing all the computations for IK and sending the results to connected viewers (reducing the amount of traffic and potential loss of messaging), it puts a potentially high compute load on the simulator (imagine the simulator trying to manage the IK for 50+ avatars at an event, tracking the movement, interactions, etc.).
A potential trade off here is to have a viewer run the IK calculations for the avatar it is controlling, and package that information for streaming to other viewers connected to the region’s simulator with minimal sanity checking (e.g. to ensure the avatar’s position in the viewer is properly constrained to within a few metres of its location as calculated by the simulator). On receipt, the sanity-checked data can then be played back without the need for the receiving viewer having to carry out IK calculations for itself on the avatar(s) it is “watching”, and carrying out some minimal sanity checks of its own.
Rider Linden is investigating having the simulator track the motion from a puppetry viewer in a way that does not impact simulator performance “too badly”. The options he’s looking at are:
Having the simulator suck the data out of the puppetry messages as they are sent through it, or
Using a new message the viewer can use to report the locations of it’s attachment points, and the simulator tracks these – which Rider sees as the preferred option.
This later method could – among other things – be expanded to work with animations in general. In addition, if tied tied in to the entire animation system (viewer computes its animation frame puppetry+legacy) to produce results, then the results could be made to scripts.
In regards to interfaces for this, Rider is of the opinion that scripts should reference attachment points since those can act as proxies for bone locations and most general creators, scripters and residents are familiar with them. and it avoids having to introduce a new concept of bones into LSL.
Collisions: the above spawned a related discussion on providing additional data such as geometry information (e.g. spherical bounding radius or a shape approximation) to allow collisions to be enabled from IK, and allowing “snap to” functionality (e.g. you reach for a glass and the avatar hand snaps to it on detecting the collision).
However, rather than allowing allow physics collisions on an attachment points (which might over-complicate the avatar model in the Havok physics engine), Rider suggested having a property (sphere) that could be set on an attachment that enabled it as a physics volume.
My thought is that collision aware attachment points, along with being able to detect and set their positions in space will be enough to get us 90% of the way towards being able to hold hands in world.
– Rider Linden
Leviathan noted there was a bug where an avatar with non-unity scale on its bones would be broken under puppetry (misalignment of bones). This should now be fixed, but it allowed him to add the theoretical ability to modify the scale of a joint in its parent-frame (an example plug-in script to demonstrate ho this works has yet to be written).
Village de Roqueblanche, October 2022 – blog post
The following notes were taken from my audio recording and chat log of the Content Creation User Group (CCUG) meeting held on Thursday, December 1st 2022 at 13:00 SLT. These meetings are chaired by Vir Linden, and their dates and times can be obtained from the SL Public Calendar.
This is a summary of the key topics discussed in the meeting and is not intended to be a full transcript.
Note: unfortunately, my recording software failed at the 50 minute mark of the meeting, so the end of the meeting and the after-meeting discussion were not recorded, so some discussion points are missing from this summary.
Official Viewers Status
Available Viewers
On Friday, December 2nd, the PBR Materials project viewer updated to version 7.0.0.576966 on Friday December 2nd. This viewer will only function on the following Aditi (beta grid) regions: Materials1; Materials Adult and Rumpus Room 1 through 4.
The following reflect the rest of the current official viewers available through Linden Lab.
Release viewer: version 6.6.7.576223 – MFA and TOS hotfix viewer – November 1 – No change.
Performance Floater / Auto-FPS project viewer, version 6.6.8.576737, November 28.
Maintenance P (Preferences, Position and Paste) RC viewer version 6.6.8.576812 on Monday, November 28.
VS 2022 RC viewer, version 6.6.8.576310, issued November 4 – utilises Visual Studio 2022 in the Windows build tool chain.
Project viewers:
Puppetry project viewer, version 6.6.3.575529, issued on October 12.
Love Me Render (LMR) 6 graphics improvements project viewer 6.6.2.573263, July 21.
General Viewer Notes
The Lab is aiming to get the Maintenance P RC viewer promoted to release status before year-end.
glTF Materials and Reflection Probes
Project Summary
To provide support for PBR materials using the core glTF 2.0 specification Section 3.9 and using mikkTSpace tangents, including the ability to have PBR Materials assets which can be applied to surfaces and also traded / sold.
To provide support for reflection probes and cubemap reflections.
The overall goal is to provide as much support for the glTF 2.0 specification as possible.
The project viewer is available via the Alternate Viewers page, but will only work on the following regions on Aditi (the Beta grid): Materials1; Materials Adult and Rumpus Room 1 through 4.
Bugs and regressions continue to be reported via the project viewer on Aditi, and it is now unlikely the project will advance to a release state before early 2023 (end of 2022 always seemed ambitious).
Despite hopes to the contrary, a lot of legacy content is impacted when rendered via the glTF PBR path, largely as a result of using linear alpha blending.
As a result, LL are working on turning to smooth around some of the edge cases without having to introduce a code fork in the viewer between “legacy” alpha handling and PBR alpha handling.
The aim remains to preserve as much of the appearance of legacy content under PBR rendering and reflection probes, without necessarily being slaved to preserving its looks over time. The major exception to this is would be situations where PBR rendering reveals seams between layers.
However, if it proves necessary, an “opt out” button will be provided to switch out of linear alpha blending.
mikkTSpace tangents affect all objects rendered in the PBR viewer. Unfortunately, due to the way mesh data has been stored prior to the PBR viewer, this means that creators wishing to get exactly the same results in their mesh models as they saw in their tool of choice (e.g. blender, etc.) when building their models, they will need to re-upload those models. This might result in a slight increase in the object’s Land Impact, if done.
This does mean, however that all tangents on uploaded meshes will be correctly handled going forward.
Capabilities will change with the PBR viewer: for example, there will be a basic glTF materials editor within the viewer; texture will rotate from a corner, rather than the centre (per the glTF specification extension for texture transforms); the upload will allow for the upload of individual materials from a glTF file.
Bakes on Mesh and Materials
Providing materials support to Bakes on Mesh has been a long-standing request which has been thus far resisted by the Lab on the grounds of the impact it would have on the Bake Service – both in terms of code updates and the potential number of servers used.
However, at the CCUG, Runitai and Vir floated the idea of materials on Bakes on Mesh being added – but only for the PBR rendering path. This would “drastically” cut down on the amount of back-end work required to make materials on BOM possible, and would mean that all avatar wearables would be Materials-capable.
This is not something that has been currently road mapped for implementation at any time by LL, and it would require time and effort to determine a mechanism to manage it, but the support were there for it, it is something that might be considered.
If this work was carried out, it might also pave the way for terrain painting – another popular request.
As PBR Materials is released, it will become the focus for SL going forward; whilst efforts will be made to ensure “pre-PBR” contact continues to look right, whether or not work is put into trying to “pull” legacy materials content into PBR is questionable, simply because the layered complexity of the underpinning code =, the updating / altering of which can result in content breakage.
In Brief
As noted in the summary of the last meeting, there are reports that the PBR project viewer generating some 10% more CPU temperature and 17% more GPU temperature.
LL are working on updates they hope will see any GPU increases return to the levels of the current release viewer.
Issues with CPU temperature are not believed to be related to PBR, but more to general texture rendering, where a couple of errors have crept in and are being actively corrected.
LODs and Land Impact:
Lower value LODs on models. Some creators attempt to “game” Land Impact by having extremely low values as their low-end models, thinking they are “never seen”; unfortunately, these are seen by people running SL on very low-end machines and thus can account for new users feeling SL “looks rubbish due to the official viewer defaulting to a RenderVolumeLODFactor of 1.25.
The RenderVoumeLODFactor setting will be more dynamic be default in the upcoming Performance Floater / Auto FPS viewer, although extremely lower LODs on models should still be avoided.
More broadly, LL is still considering how to better adjust the Land Impact system and the mesh uploader so that creators are not penalised for creating accurate LODs on their models (one of the goals originally stated for project ARCTan).
Linden Lab will be closed over the holiday period from end of business on Friday, December 23rd, 2022 through until start of business on Tuesday, January 3rd, 2023 (except for urgent support cases).
Inara Pey: Living in a Modemworld 