Day: May 12, 2023

2023 SL Puppetry project week #19 summary

Posted on by Inara Pey
Puppetry demonstration via Linden Lab – see below.  Demos video with the LL comment “We have some basic things working with a webcam and Second Life but there’s more to do before it’s as animated as we want.”

Note: this has been updated to include comments made at the TPV Developer meeting on Friday, May 12th.

The following notes have been taken from chat logs and audio recording of the Thursday, May 11th, 2023 Puppetry Project meetings held at the Castelet Puppetry Theatre on Aditi. These meetings are generally held on alternate weeks to the Content Creation User Group (CCUG), on same day / time (Thursdays at 13:00 SLT).

Notes in these summaries are not intended to be a full transcript of every meeting, but to highlight project progress / major topics of discussion.

Project Summary

General Project Description as Originally Conceived

LL’s renewed interest in puppetry was primarily instigated by Philip joining LL as official advisor, and so it really was about streaming mocap. That is what Philip was interested in and why we started looking at it again. However since Puppetry’s announcement what I’ve been hearing from many SL Residents is: what they really want from “puppetry” is more physicality of the avatar in-world: picking up objects, holding hands, higher fidelity collisions. 
As a result, that is what I’ve been contemplating: how to improve the control and physicality of the avatar. Can that be the new improved direction of the Puppetry project? How to do it?

Leviathan Linden

  • Previously referred to as “avatar expressiveness”, Puppetry is intended to provide a means by which avatars can mimic physical world actions by their owners (e.g. head, hand, arm movements) through tools such as a webcam and using technologies like inverse kinematics (IK) and the  LLSD Event API Plug-in (LEAP) system.
    • Note that facial expressions and finger movements are not currently enabled.
    • Most movement is in the 2D plain (e.g., hand movements from side-to-side but not forward / back), due to limitations with things like depth of field tracking through a webcam, which has yet to be addressed.
  • The back-end support for the capability is only available on Aditi (the Beta grid) and within the following regions: Bunraku, Marionette, and Castelet.
  • Puppetry requires the use of a dedicated viewer, the Project Puppetry viewer, available through the official Second Life Alternate Viewers page.
  • No other special needs beyond the project viewer are required to “see” Puppetry animations. However, to use the capability to animate your own avatar and broadcast the results, requires additional work – refer to the links below.
  • There is a Puppetry Discord channel – those wishing to join it should contact members of LL’s puppetry team, e.g. Aura Linden, Simon Linden, Rider Linden, Leviathan Linden (not a full list of names at this time – my apologies to those involved whom I have missed).

Additional Work Not Originally In-Scope

  • Direct avatar / object / avatar-avatar interactions (“picking up” an apple; high-fives. etc.).
  • Animations streaming: allowing one viewer to run animations and have them sent via the simulator to all receiving viewers without any further processing of the animations by those viewers.
  • Enhanced LSL integration for animation control.
  • Adoption of better animation standards – possibly glTF.
  • Given the project is incorporating a lot of additional ideas, it is likely to evolve into a rolling development, with immediate targets for development / implementation decided as they are agreed upon, to be followed by future enhancements. As such, much of what goes into the meetings at present is general discussion and recommendations for consideration, rather than confirmed lines o development.

Bugs, Feature Requests and Code Submissions

  • For those experimenting with Puppetry, Jiras (bug reports / fixes or feature requests) should be filed with “[Puppetry]” at the start of the Jira title.
  • There is also a public facing Kanban board with public issues.
  • Those wishing to submit code (plug-ins or other) or who wish to offer a specific feature that might be used with Puppetry should:

Further Information

Meeting Notes

Viewer Progress

  • An updated version of the viewer – version 6.6.12.579958 – was release on Thursday, May 11th.
    • This update includes access to Rider Linden’s experimental attachment point tracking & forwarding to the server feature.
    • It also includes various incremental improvements to handling puppetry, such as support for parsing binary LEAP data from the LEAP script.
  • Avatar attachment point tracking (per the TPVD meeting discussion on May 12th):
    • This allows the tracking of joints (using attachment points) using a script.
    • Using visible attachment points (i.e. those on the avatar , NOT any of the screen-based HUD attachment points) cuts down on the the amount of data having to be be handled at both ends.
    • The speed at which the attachment point movement is read back is such that it could not be exploited to create a copy of an animation with any real fidelity.
    • This is a deliberate move to ensure that animation creators are not left feeling uncomfortable about LSL animation tracking.
    • There is a combined throttle / sleep time elements to tracking attachment points: the throttle limits the number of attachment points that can be tracked over a certain period of time; he script sleep time is designed to allow an animation to move those attachment points forward sufficiently before a further taking record is made. Thus, it is next to impossible to track and record a coherent animation frame.
  • It was noted that previously, joint constraints had been hard coded in C++, but their configuration has been moved into a human-readable LLSD file which can be modified without rebuilding the viewer.
  • Two areas of focus going forward are:
    • Improving the Inverse Kinematic (IK) system within the viewer – something Leviathan Linden is already working on. This will include overall improvements to IK constraints as well as to positioning, with the existing source-code constraints moved  replaced by a further config file – “constraints” here being in terms of joint rotation / movement.
    • Providing .FBX animation file import and Mixamo skeleton re-targeting.
  • The IK work is still being thrashed out (and subject to much more discussion at meetings, but is seen as a priority over other elements of work, such as the animations streaming idea Leviathan Linden had been working on. The hope is that by improving IK, it will play into streaming and “live” animations a lot more robustly and smoothly. It is also seen as a foundational piece of work for further opening up puppetry and animation work.

General Notes

  • It was noted that for animation import, LL is looking towards using / supporting Assimp (github: https://github.com/assimp/assimp), which supports some 30-40 animation formats, converting them to it ownformat for ease of import to multiple platforms. Notably, it supports .FBX and glTF, so it fits with the Lab’s goal of utilising glTF for materials, mesh imports, etc.
  • [TPVD meeting, May 12th] This will not alter the existing internal format for animation. It is just  to allow the import of other formats.
  • It is acknowledged that alongside o that, the Labe will require a retargeting system for animations; although what form this will take is still TBD.
  • The core of the meeting was a general discussion of things that might / could e done in the future, and what technologies LL might look towards.

Date of Next Meeting

  • Thursday, May 25th, 2023, 13:00 SLT.

Intimist journeys and Zorian women in Second Life

Posted on by Inara Pey
Nitroglobus Roof Gallery: Nowhere B. – Intimistic Journey

May has brought with it not one but two exhibitions at Nitroglobus Roof Gallery, curated by Dido Haas. Whilst both very different in tone and content, they are nevertheless connected in that each has a particular source of inspiration / interpretation.

Occupying the Annex section of the Gallery, and having opened on May 2nd, is a compact exhibition which should be noted as potentially Not Suitable For Work (NSFW), given the subject is nude studies. Entitled Zorian Women, it presents a total of 15 studies of the female form, as imaged by Kian (random26356). Fourteen of these (the 15th is rather more novel in approach!), have been inspired by the nude studies of “the Swedish Impressionist”, Anders Leonard Zorn (18 February 1860 – 22 August 1920).

Nitroglobus Roof Gallery: Kian – Zorian Women

Zorn was an incredibly gifted artist, a veritable virtuoso working in multiple formats and styles – watercolours, oils, pencil, etching, sculpture – to produce an incredible range of art during his lifetime. A graduate of the Royal Swedish Academy of Arts, at the age of 29 he was made a Chevalier de la Légion d’honneur at the 1889 World Fair in Paris, and he would gain international recognition for his portraiture, with subjects including three US Presidents and King Oscar II of Sweden, whilst he and his wife Emma would go on to established The Bellman Prize for Swedish poetry, which is still awarded annually by Swedish Academy.

Zorn’s repertoire included a series of female nude studies which became particularly noted for their unique style. Initially produced in watercolours, these depicted his models fully unclothed and painted against the backdrop of the open countryside of  Dalarna, the district where he and Emma lived in Sweden. Such was the form and impact of the pieces in this collection, they became known under the collective name of Zornkulla. 

Nitroglobus Roof Gallery: Kian – Zorian Women

It is these works Kian celebrates within his exhibition. This is no easy task, because despite their apparent casualness of presentation, employing a loose brushwork, soft tones and gently blurred edges, the Zornkulla paintings are all incredibly well composed through use of colour, pose, placement of elements such as water and the horizon and features within the landscape beyond the nude subject, all of which results in a visual play: focus on the subject, and she immediately comes into sharp focus, the background dissolving so as not to be noticed. But then focus on the backdrop, and the subject herself appears to dissolve into it, becomes an almost natural part of the landscape, rather than appearing as a something standing within it.

Emulating this technique, Kian offers that same sense of loose brushwork and familiar soft tones whilst adding his own interpretation on Zorn’s Zornkulla, thus offering something fresh and new.

Within the main halls of Nitroglobus is the first solo exhibition by Nowhere B. A relative newcomer to SL photography and to the art exhibition scene here – his first time exhibition was at an ensemble event in 2021 – there could not be a better, more nurturing environment than Nitroglobus under Dido’s stewardship, by which he could spread his wings into the world of solo exhibitions.

Nitroglobus Roof Gallery: Nowhere B. – Intimistic Journey

Entitled Intimistic Journey, this is an engaging and somewhat personal selection of pieces by Nowhere B., representing as it does a record of his journeys through Second Life’s richly varied landscapes and settings, whilst also offering hints of his inner responses and experiences during those visits (hence the intimist sub-text to the title).

Given this, these are not you usual captures of places seen in Second Life. Instead they offer a highly personalised perspective (in places quite literally!) and narrative to present not just reflections on the places Nowhere visited, but his potential mood and thoughts on seeing them. This is helped in part by the artist including himself – and his John Steed-like bowler and brolly – in many of the images in a manner that draws us into his thinking without bashing u over the held with obviously posed context and narrative.

Nitroglobus Roof Gallery: Nowhere B. – Intimistic Journey

Taken individually or together, Zorian Women and Intimistic Journey present engaging, highly visual and expressive exhibitions which can be enjoyed both independently to one another, or as a joint visit to Nitroglobus, with Zorian Women set to remain in place through into the better part of June.

SLurl Details