Rosehaven Seafolk Sanctuary, August 2022 – blog post
The following notes were taken from the Tuesday, September 20th, 2022 Simulator User Group (SUG) meeting. They form a summary of the items discussed and is not intended to be a full transcript. A video of the entire meeting is embedded at the end of the article for those wishing to review the meeting in full – my thanks to Pantera for recording it.
Server Deployments
At the time of writing, there was no published deployment plan for the week.
On Tuesday, September 27th, the simhosts on the Main SLS channel were restarted with no deployment, leaving them on simulator release 574921, comprising the fixes from release 574611 + BUG-232593 “Mesh physics shapes changing unexpectedly when rezzed on a 574611 sim”, introduced with that update.
On Wednesday, September 28th, the simhosts on the RC channels should receive an update containing two new functions llGetObjectLinkKey (specified under llGetLinkKey) and llSHA256String.
Available Official Viewers
No changes to mark the start of the week, leaving the official viewer pipelines as:
Release viewer: version 6.6.4.575022 – hotfix for Crash at ~LLModalDialog() – promoted September 15 – NEW.
Puppetry demonstration via Linden Lab – see below. Demos video with the LL comment “We have some basic things working with a webcam and Second Life but there’s more to do before it’s as animated as we want.”
The following notes have been taken from chat logs and audioo recordings of the September 8th and September 22nd Puppetry Project meetings held at the Castelet Puppetry Theatre on Aditi. These meetings are:
Generally held on alternate weeks to the Content Creation User Group (CCUG), on same day / time (Thursdays at 13:00 SLT).
A mixed Voice / text chat format – attendees are not obligated to use voice when asking questions, but will need to listen to voice to hear the entire meeting.
Notes in these summaries are not intended to be a full transcript of every meeting.
Project Summary
Previously referred to as “avatar expressiveness”,
Puppetry is intended to provide a means by which avatars can mimic physical world actions by their owners (e.g. head, hand, arm movements) through tools such as a webcam and using technologies like inverse kinematics (IK) and the LLSD Event API Plug-in (LEAP) system.
Note that facial expressions and finger movements are not currently enabled.
Most movement is in the 2D plain (e.g., hand movements from side-to-side but not forward / back), due to limitations with things like depth of field tracking through a webcam, which has yet to be addressed.
The back-end support for the capability is only available on Aditi (the Beta grid) and within the following regions: Bunraku, Marionette, and Castelet.
Puppetry requires the use of a dedicated viewer, the Project Puppetry viewer, available through the official Second Life Alternate Viewers page.
No other special needs beyond the project viewer are required to “see” Puppetry animations. However, to use the capability to animate your own avatar and broadcast the results, requires additional work – refer to the links below.
There is now a Puppetry Discord channel – those wishing to join it should contact members of LL’s puppetry team, e.g. Aura Linden, Simon Linden, Rider Linden, Leviathan Linden (not a full list of names at this time – my apologies to those involved whom I have missed).
For those experimenting with Puppetry, Jiras (bug reports / fixes or feature requests) should be filed with “[Puppetry]” at the start of the Jira title.
Those wishing to submit code (plug-ins or other) or who wish to offer a specific feature that might be used with Puppetry should:
Discuss them with the Puppetry team: and work with them to ensure a proper convergence of ideas.
Be signed-up to the Lab’s contribution agreement in order for submitted code to be accepted for review / use:
Note: timing issues on my part meant I was unable to attend the first third of this mssting.
It is acknowledged that the current Puppetry viewer (viewer branch DRTVWR-558) is somewhat crashy and subject to some looping issues.
One aspect of Puppetry that should be highlighted is the ability for it to work alongside / in concert with existing SL animations – so you can be running a dance animation and still wave to a friend using puppeteering without the two animations clashing.
It is acknowledged that to ensure some reasonable smoothness of movement and to prevent things like movement conflicts between joints, there will need to be a more formalised animation constraints system. The current plan is to make this configurable via XML.
It is also acknowledged that tracking in general needs to be tightened within the plug-in code.
Puppetry does not currently interact with the Havok physics system (puppetry is largely viewer-side; physics – with the exception of some special use sub-libraries – is largely simulator-side).
The protocols which are used server-side to support Puppetry are not set in stone at this point; cases which require additional messaging, etc. can be discussed with the Puppetry team members from the simulator / server side of LL (e.g. Rider and Simon Linden).
Direct avatar interactions (e.g. shaking / holding hands, swinging a tennis racket to strike a ball, etc.): the IK system could help enable this, but it would also require a lot more work on the avatar / world mapping system to be fully possible, and this work has yet to be tackled (if it is to be tackled as a part of this initial Puppetry work).
The project is, at this point, fairly open as to where it might go: these initial project meetings are geared towards developers who may be interested in contributing and pushing elements of the project forward (e.g. support for full body tracking, etc.). Obviously, at some point, constraints will be placed on what is to be initially delivered.
Plugins (Pros and Cons)
Requests were made for the Puppetry system to support OpenXR (as well as LEAP). It was indicated that OpenXR would be considered as a default if a suitable plug-in were to be developed and contributed to Linden Lab for proper vetting and formal inclusion in the viewer.
The fact that the Puppetry project is using plug-ins raised concerns over system security. Plug-ins are executable, and so if accepted to run, a malicious plug-in could do considerable harm to a person’s system.
LL is aware of this, and is actively trying to minimise risk as far as possible.
However, safety also lay with users – do not download viewers from unofficial sites / sites that cannot be trusted; do not accept and run plug-ins that are passed around through forums, etc.
The benefits of using plug-ins was summarised as:
Speed of internal development / testing: there is no need to run a complete viewer build process simply because a couple on lines of code have been changed in testing; only the plug-in needs to be updated.
Extensibility: plugs-ins allow for more flexible support of additional creation tools or to add support for additional data formats (e.g. as with OpenXR) / hardware / programming languages (e.g. Python, C++, etc.).
Performance: using plug-ins allows the required additional processing such as webcam capture, processing and translation to be handed-off the separate processing threads within a computer from the viewer, thus preventing the latter losing performance by having to do the processing itself.
User assurance: removing things like the webcam controls to a plug-in that is not run by default as a de facto part of the viewer’s processing will (hopefully) remove fears about webcams somehow being used to “spy” on users.
Summary of September 22nd Meeting
It is hoped an updated version of the Puppetry Project Viewer will be available via the Alternate Viewers page in week #39 (commencing Monday, September 26th). This includes fixes and updates to the motion logic that should make avatar motion more predictable.
In terms of device support for puppeteering, any device that can be recognised as a joystick should be supportable within the Puppetry viewer (utilising the existing Joystick support options through Preferences) – although some refinement to the controls may be required via LL.
LSL support for puppeteering: nothing has been defined at present, but there are some ideas as to what might be needed / nice to have. It has been suggested LSL support is a subject for discussion at the next meeting.
Simon Linden has pushed a couple of capabilities:
A simple poser contained in a side branch of the LEAP repository. This reads a basic JSON file with bone positions (rotations) for all 133 bones in the avatar skeleton and sends it as LEAP data to the viewer for animating the avatar. Thisfile can be live-edited, and is desgined to help those working with puppeteering to experiment with it in an easy format – it will not be an end feature for the project.
Added a further branch to the Puppetry viewer repository called DRTVWR-558 Data Packing. This converts the data going from the viewer to the server onwards to a more efficient format, allowing the full animation data set to be contained in a single packet for transmission.
However, this format is incompatible with the existing data format used within viewers built via DRTVWR-558; so as viewers are built using the newer code, this will not be able to show puppeteering using the older format, and vice-versa.
Those involved in experimenting with Puppetry should therefore switch to the viewer using the updated data format, once this is made available through the Alternate Viewer page, as it will be replacing the current data format going forward.
Leviathan Linden has suggested that if LL can transmit all bone data in compressed format, then they may not need to send IK targets and have the viewer manage the IK for all avatars in a scene, but rather have the viewer run the IK for a user’s avatar and then stream the avatar’s entire state, reducing the load on the viewer.
Pelvis Movement / Full Body Tracking / OpenXR Support
There was initial discussion about supporting local joint offsets and particularly off-setting the avatar pelvis to allow for subtle movements without actually moving the avatar.
This is somewhat similar to scripted animations, such as stands with an AO system – the avatar appears to step forward / back / walk in circle, but it is not physically moving as far as the simulator is concerned – the motions are the result of the avatar pelvis being offset from it’s actual position as seen by the simulator, and the animations running based on that offset.
There was some initial confusion over this and physically moving the avatar, as such, it was suggested this be referred to as “pelvis movement, rather than “offsetting joints / bones”.
Part of the reason for this discussion is because several non-Linden developers have been experimenting with partial and full-body tracking via OpenXR, and have found that not being able to move the pelvis within Puppetry can lead to issues of floating, etc., when an avatar kneels or crouches (as seen within existing SL animations) – the result of the legs being pulled up towards the pelvis, rather than the pelvis being moved towards the ground.
In addition this work has noted:
If Second Life were to return the “full” appearance data for an avatar (i.e. after allmesh transforms, slider data,, baked appearance information, etc.) has been applied, rather than the “raw” skeletal appearance, better calculations could be made around the pelvis height from the floor.
The approach works equally well with partially body tracking via a Rift S headset, and fully body tracking using alve headsets and Kinect devices.
However, it currently uses Blender as a conduit for translating movement within an OpenXR rig to the Second Life puppeteering rig, and would benefit enormously from a dedicated OpenXR plug-in, and the developers are willing to provide data data gathered from the work they’ve thus far completed to help facilitate this.
Separately to this, OPEN-363 “[Puppetry] [LEAP]: Add native OpenXR plugin” has been raised, but is (at the time of writing) awaiting review.
The above formed a nucleus of the discussion for much of the meeting with the ability to move the avatar pelvis now being seen as more of a priority requirement, with Leviathan Linden indicating they will try to look specifically at this between now and the next meeting.
Date of Next Meeting
Thursday, October 13th, 2022 – but check the SL public calendar to confirm.
The following notes were taken from the Tuesday, September 20th, 2022 Simulator User Group (SUG) meeting. They form a summary of the items discussed and is not intended to be a full transcript. A video of the entire meeting is embedded at the end of the article for those wishing to review the meeting in full – my thanks to Pantera for recording it.
On Tuesday, September 20th, the simhosts on the Main SLS channel were updated with simulator release 574921, comprising the fixes from release 574611 + BUG-232593 “Mesh physics shapes changing unexpectedly when rezzed on a 574611 sim”, introduced with that update.
On Wednesday, September 21st, the simhosts on the RC channels will be restarted without any deployment, leaving them on simulator release 574921.
Available Official Viewers
On Monday, September 19th, Maintenance P RC viewer updated to version 6.6.5.575055.
The rest of the current official viewers remain as:
Release viewer: version 6.6.4.575022 – hotfix for Crash at ~LLModalDialog() – promoted September 15.
Release channel cohorts:
Maintenance 3 RC viewer, version 6.6.4.574727, September 1.
Project viewers:
Puppetry project viewer, version 6.6.3.574545, issued on August 30.
Love Me Render (LMR) 6 graphics improvements project viewer 6.6.2.573263, July 21.
Performance Floater project viewer, version 6.5.4.571296, May 10.
In Brief
The next simulator maintenance release will have two new functions llGetObjectLinkKey (specified under llGetLinkKey) and llSHA256String. These should be available for testing on “a few regions” (no names specified) on Aditi by Thursday, September 23rd.
There are two further feature requests for an llHash function – BUG229023 and BUG-41440.
A further dicussion on region crossings, initially focused on BUG-230934.
The recent Mediaon a Prim (MOAP) exposures were raised at the meeting, together with the general prboem of MoAP vulnerabilities – but given media handling is viewer-centric, there was little to be said from the Lab’s side in terms of addressing problems (although the upcming tool chain upates for the viewer should at least allow LL to adopt more recent and better secured Chrome Embedded Framework (CEF) versions).
The following notes were taken from the Tuesday, September 13th, 2022 Simulator User Group (SUG) meeting. They form a summary of the items discussed and is not intended to be a full transcript. A video of the entire meeting is embedded at the end of the article for those wishing to review the meeting in full – my thanks to Pantera for recording it.
On Tuesday, September 13th, the simhosts on the Main SLS channel were re-started without any deployment, leaving them on simulator version 574550.
On Wednesday, September 14th, all simhosts on the RC channels will receive a fix for BUG-232593 “Mesh physics shapes changing unexpectedly when rezzed on a 574611 sim”. This fix should not involve any other changes to the simulator version.
In short, there was a change that attempted to correct some bounding box calculations and in a couple cases it would wildly change the mass and centre of gravity.
Available Official Viewers
On Tuesday, September 13th, the Izarra Maintenance RC, version 6.6.4.574885 (dated September 9th) was promoted to de facto release status.
The rest of the current official viewers remain as:
Maintenance P (Preferences, Position and Paste) RC viewer version 6.6.4.574750, issued September 6.
Maintenance 3 RC viewer, version 6.6.4.574727, September 1.
Project viewers:
Puppetry project viewer, version 6.6.3.574545, issued on August 30.
Love Me Render (LMR) 6 graphics improvements project viewer 6.6.2.573263, July 21.
Performance Floater project viewer, version 6.5.4.571296, May 10.
In Brief
Display names not updating correctly (see this grid status update): this may be related to a back-end server issue, rather than a table update issue, but is apparently still under investigation.
A general discussion on scripted access to block lists – although the use case wasn’t clearly explained (e.g. scripted access to one’s own block list – or others of others (and if so why & what about the potential for breach of privacy?)). This is now pending a feature request to clarify.
BUG-231802 “Prevent vehicles from entering parcels their riders cannot access”, together with providing a means for a viewer to request all active ban lines within a region – work still pending, and Leviathan Linden (the Linden Formerly Known As Andrew) offer some insights into the complexities of managing parcel ban collisions, which leads to further discussion wound into other topics.
BUG-232468 “[Feature Request] llGetObjectLinkKey() – A remote equivalent of llGetLinkKey()” – may well by in the next-but one (or two) simulator maintenance update.
The following notes were taken from the Tuesday, September 6th, 2022 Simulator User Group (SUG) meeting. They form a summary of the items discussed and is not intended to be a full transcript. A video of the entire meeting is embedded at the end of the article for those wishing to review the meeting in full – my thanks to Pantera for recording it.
On Tuesday, September 6th, the simhosts on the Main SLS channel were re-started without any deployment, leaving them on simulator version 574550.
On Wednesday, September 7th, all simhosts on the RC channels will likewise be restarted without any deployment, leaving them on simulator release 574611.
Available Official Viewers
On Tuesday, September 6th, the Maintenance P (Preferences, Position and Paste) RC viewer updated to version 6.6.4.574750.
The rest of the current official viewers remain as:
Release viewer: version 6.6.3.574158 – formerly the Profiles RC viewer, dated August 18, promoted August 30 – NEW.
Puppetry demonstration via Linden Lab – see below. Demos video with the LL comment “We have some basic things working with a webcam and Second Life but there’s more to do before it’s as animated as we want.”
The following notes were taken from the Tuesday, August 30th, 2022 Simulator User Group (SUG) meeting. They form a summary of the items discussed and is not intended to be a full transcript. A video of the entire meeting is embedded at the end of the article for those wishing to review the meeting in full – my thanks to Pantera for recording it.
On Tuesday, August 30th, the simhosts on the Main SLS channel were updated with simulator version 574550. This release fixes an issue with some system textures being unavailable – notably the default water map being absent, resulting in totally flat glassy water or view-transparent mode having everything look solid..
On Wednesday, August 31st, all simhosts on the RC channels will be updated with simulator release 574611,which includes the above fixes + additional fixes (release notes pending at the the time of writing.
Available Official Viewers
On Tuesday, August 30th:
The Profiles viewer, 6.6.3.574158, was updated to de facto release status.
The Puppetry project viewer, version 6.6.3.574545, was issued – see below.
The remaining official viewers stay listed as:
Release channel cohorts:
Maintenance P (Preferences, Position and Paste) RC viewer version 6.6.3.573877 issued August 15.
Izarra Maintenance RC, version 6.6.3.573920, August 15.
Maintenance 3 RC viewer, version 6.6.3.573882, August 5.
Project viewers:
Love Me Render (LMR) 6 graphics improvements project viewer 6.6.2.573263, July 21.
Performance Floater project viewer, version 6.5.4.571296, May 10.
Previously referred to as “avatar expressiveness”, Puppetry is intended to provide a means by which avatars can mimic physical world actions by their owners through tools such as a webcam and using technologies like inverse kinematics and the LLSD Event API Plug-in (LEAP) system.
We are excited about Puppetry’s potential to change the way we interact inside Second Life. For example, using a webcam to track your face and hands could allow your avatar to mimic your face animations and finger movement, or more natural positioning of the avatar’s hands and feet against in-world objects might also be possible. Alternative hardware could be used to feed information into Second Life to animate your avatar – a game controller or mocap equipment. There’s a lot to explore and try, and we invite the Second Life community to be involved in exploring the direction of this feature.
– from the official blog post
Note that this capability is still very much experimental / in development and is not feature-complete. Also, the following should be noted:
The Puppetry viewer is very much experimental, and is required to view any puppetry animations – viewers without the supporting code will not show them.
The back-end support for the capability is only available on Aditi (the Beta grid) and within the following regions: Bunraku, Marionette, and Castelet.
No other special needs beyond the project viewer are required to “see” Puppetry animations. However, to use the capability to animate your own avatar and broadcast the results will take more work.
For further information, check out the documentation links below:
Again, please remember this is very much a “alpha” implementation of Puppetry:
Linden Lab is looking for feedback from the community in determining potential use cases and onward development.
The Lab will be hosting an open discussion on Puppetry on Thursday, September 8th, and 13:00PM SLT. This will take place at the Puppetry theatre in the Aditi region of Castelet.
Please refer to the video for further discussion on the capability.
Update, September 1st
The discussion on puppetry mentioned in the above articles will be the first such meeting, and if there is demand for it, there will be a similar meeting on Aditi on alternate Thursdays from September 8th onwards, to be held in the theatre on Aditi Castelet region.
These meetings will (initially) be very development focused rather than creator / user focused, given the overall status of the project.
It is advisable that attendees use the Puppetry project viewer when attending these meetings (available from the Alternate Viewers page), so that they might see any demonstration which may take place during meetings.
Inara Pey: Living in a Modemworld 