Day: February 24, 2023

2023 SL Puppetry project week #8 summary

Posted on by Inara Pey
Puppetry demonstration via Linden Lab – see below.  Demos video with the LL comment “We have some basic things working with a webcam and Second Life but there’s more to do before it’s as animated as we want.”

The following notes have been taken from chat logs and audio recording of the Thursday, February 24th, 2023 Puppetry Project meetings held at the Castelet Puppetry Theatre on Aditi. These meetings are generally held on alternate weeks to the Content Creation User Group (CCUG), on same day / time (Thursdays at 13:00 SLT).

Notes in these summaries are not intended to be a full transcript of every meeting, but to highlight project progress / major topics of discussion.

Project Summary

General Project Description as Originally Conceived

LL’s renewed interest in puppetry was primarily instigated by Philip joining LL as official advisor, and so it really was about streaming mocap. That is what Philip was interested in and why we started looking at it again. However since Puppetry’s announcement what I’ve been hearing from many SL Residents is: what they really want from “puppetry” is more physicality of the avatar in-world: picking up objects, holding hands, higher fidelity collisions. 
As a result, that is what I’ve been contemplating: how to improve the control and physicality of the the avatar. Can that be the new improved direction of the Puppetry project? How to do it?

Leviathan Linden

  • Previously referred to as “avatar expressiveness”, Puppetry is intended to provide a means by which avatars can mimic physical world actions by their owners (e.g. head, hand, arm movements) through tools such as a webcam and using technologies like inverse kinematics (IK) and the  LLSD Event API Plug-in (LEAP) system.
    • Note that facial expressions and finger movements are not currently enabled.
    • Most movement is in the 2D plain (e.g., hand movements from side-to-side but not forward / back), due to limitations with things like depth of field tracking through a webcam, which has yet to be addressed.
  • The back-end support for the capability is only available on Aditi (the Beta grid) and within the following regions: Bunraku, Marionette, and Castelet.
  • Puppetry requires the use of a dedicated viewer, the Project Puppetry viewer, available through the official Second Life Alternate Viewers page.
  • No other special needs beyond the project viewer are required to “see” Puppetry animations. However, to use the capability to animate your own avatar and broadcast the results, requires additional work – refer to the links below.
  • There is a Puppetry Discord channel – those wishing to join it should contact members of LL’s puppetry team, e.g. Aura Linden, Simon Linden, Rider Linden, Leviathan Linden (not a full list of names at this time – my apologies to those involved whom I have missed).

Additional Work Not Originally In-Scope

  • Direct avatar / object / avatar-avatar interactions (“picking up” an apple; high-fives. etc.
  • Animations streaming: allowing one viewer to run animations and have them sent via the simulator to all receiving viewers without any further processing of the animations by those viewers.
  • Enhanced LSL integration for animation control.
  • Adoption of better animation standards – possibly glTF.
  • Given the project is incorporating a lot of additional ideas, it is likely to evolve into a rolling development, with immediate targets for development / implementation decided as they are agreed upon, to be followed by future enhancements. As such, much of what goes into the meetings at present is general discussion and recommendations for consideration, rather than confirmed lines o development.

Bugs, Feature Requests and Code Submissions

  • For those experimenting with Puppetry, Jiras (bug reports / fixes or feature requests) should be filed with “[Puppetry]” at the start of the Jira title.
  • There is also a public facing Kanban board with public issues.
  • Those wishing to submit code (plug-ins or other) or who wish to offer a specific feature that might be used with Puppetry should:

Further Information

Meeting Notes

General Progress

  • LSL Integration:
    • See: OPEN-375: “LSL Functions for reading avatar animation positions”.
    • Rider Linden has not been able to get a lot done on the scripted control due to being out of the office. He does have the LSL function discussed in the last meeting so that it is correctly sending the necessary data down to the agent’s viewer.
    • He is now working now on how to feed that into the IK, and has a general framework, although he notes it’s been slow progress.
  • Simon Linden has been working on animation importing. This is additional work in terms of the Puppetry project, but comes as a result of discussions at previous meetings.
    • He is looking to add additional .BVH support, and possibly .FBX (e.g. .FBX using some specific skeletons and settings; the goal is to be able to get data out of animation tools and into SL without requiring 2 years of Blender skills). Given the general move towards glTF, this is seen as being more preferable (there is a possible appetite within LL for a re-write of the animation system, although it not on the immediate horizon (or a visible horizon at present).
    • Requests are still being made to allow animation priorities to be changed post-upload and edit animation values dynamically – it is not clear how much of this will be touched.
    • Changing the manner in which animation priorities currently work is not something LL are planning on touching.
    • Right now the messages that transmit what animations to play do not have a way to specify a priority, just the animation’s asset ID and the viewer will get the priority from the asset. This may change in the future, but the focus right now is on getting scripted animation control improved.
  • Leviathan Linden is continuing to work on animation streaming, but progress has been delayed due to bug hunting and fixing. However, he hopes to get the code into the Puppetry project viewer branch sooner rather than later. He has noted that this is very sensitive to bad framerate on the sender and on the simulator. This probably means that before animation streaming and/or puppetry could be “delivered”, some technical debt on the server at least.
  • The focus at the moment is on putting everything that has been worked on together and then making sure it all works within the viewer. After that comes the issue of making sure that things work between viewers (e.g. that 20 people running animation streaming in a scene does not result in the viewers collapsing or being unable to playback all the streams; ensuring the new capabilities paly nicely with existing “canned animation” systems (e.g. dance machines, etc.).

In Brief

  • It’s been noted that moving the simulators to 64-bit is being worked on.

Date of Next Meeting

  • Thursday, March 9th, 2023, 13:00 SLT.

Of the sea and the stars in Second Life

Posted on by Inara Pey
Sur-Mer / LunarVille, February 2023 – click any image for full-size

Shawn Shakespeare recently dropped me the LM to LunarVille / Sur-Mer. As the name suggests, this is setting of two parts. joined by a common theme which might be said to just touch on global warming:

Earth circa 2052.
All continents have been submerged by water, most life is extinct. The 1% have expatriated to Mars and the lunar surface. Some survivors have managed to survive and harvest the seas, they have scrapped all vintage and advanced technology.

From Sur-Mer and LunarVille

Sur-Mer / LunarVille, February 2023

The Landing Point offers advice on viewing the settings (in short: make sure you have Used Shared Environment active in your viewer) and a choice of teleport destinations The Hangar, Sur-Mer and LunarVille, all of which can be accessed via the teleport disk.

Which you visit first is entirely up to you; I opted to hop to Sur-Mer to start my visit; a place where the last remnants of human civilisation on Earth might be found, huddled along the span of an ancient bridge, surrounded by cold-looking water and a shroud of mist. Visitors arrive on a small deck below and to one side of the bridge, photographs on a panel showing what might have happened to Earth in our future history.

Sur-Mer / LunarVille, February 2023

It is a strange place where the bridge runs from a cut-open diesel loco has been converted into a stage show setting watched by a small audience and the bridge runs to a most unusual home. Given that technology both vintage and advanced have been scrapped, there are robots to be found, powered islands hovering over the waters and home to garden or wind turbines – and even a WW2 era Mustang fighter apparently coming into land (complete with its undercarriage arranged to resemble that of a Spitfire), while at the house a line of flying vehicles are swooping in for a landing.

Strange it might be, but the location is genuinely photogenic for avatar and artistic shots, whilst the scattered seating present plenty of opportunities to set and relax.

Sur-Mer / LunarVille, February 2023

LunarVille presents an equally intriguing setting. Central to it is the Moonbase where – presumably – some of the top 1% are living. It is certainly a cosy place and comes complete with its own nightclub – The Apollo Lounge. A second dance area is available within a hanger space – although the DJ will need a spacesuit.

Scattered around the rest of this setting are multiple points of interest for explorers – the module containing the teleport, the lunar equivalent of Area 51, a small meadow sitting under a dome where a cow and sheep graze and more, all of which offer further opportunities for photography.

Sur-Mer / LunarVille, February 2023

Unusual and curled into fun with a hint of a message, LunarVille / Sur-Mer makes for a light and quirky visit.

SLurl Details