Category: Content Creation User Group

2023 week #29: SL CCUG meeting summary: Senra, glTF, etc.

Posted on by Inara Pey
LemonCliff, May 2023 – blog post †

The following notes were taken from my audio recording and chat log transcript of the Content Creation User Group (CCUG) meeting held on Thursday, July 20th, 2023. 

  • The CCUG meeting is for discussion of work related to content creation in Second Life, including current work, upcoming work, and requests or comments from the community, together with viewer development work.
  • As a rule, these meetings are:
    • Held in-world and chaired by Vir Linden.
    • Conducted in a mix of voice and text.
    • Held at 13:00 SLT on their respective days.
    • Are subject to the schedule set within the SL Public Calendar, which includes locations for both meetings (also included at the end of these reports).
    • Open to all with an interest in content creation.

Viewer Updates

  • The Maintenance U(pbeat) RC viewer, version 6.6.14.581101, was released on July 21st. Key changes in this viewer comprise:
    • Improvement parcel audio as the viewer leverages VLC for audio streams.
  • The Inventory Extensions viewer was promoted to RC status with version 6.6.14.581058, on July 20th.
    • A new option Show Ban Lines On Collision (toggled via World→Show) which will only show banline on a direct collision (foot or vehicle) rather than constantly visible when within camera range.
  • The Alternative Viewers page appears to have suffered a hiccup, listing version 6.6.12.579987 as the “Win32+MacOS<10.13” RC viewer.  However,
    • The Win 32  + Pre-MAC OS 10.3 viewer was promoted to release status on July 5th.
    • 6.6.12.579987  was the version umber assigned to the Maintenance S RC viewer (primarily translation updates), originally issued on May 11th, and promoted to de facto release status on May 16th.

The release and Project viewers currently in the pipeline remain unchanged:

    • Release viewer: 6.6.13.580918, formerly the Maintenance T RC viewer, promoted on July 14.
    • Project viewers:

Senra NUX Avatars

  • There was a stir in the week when the Senra brand of mesh avatars designed by LL (and primarily intended for new users as a part of the New User eXperience  – NUX) were made available through the system Library and then withdrawn.
  • This was apparently not an error on LL’s part, but rather the result of an issue with the avatars being noted, prompting their removal from the Library.
  • The removal did not prevent some users grabbing copies of the avatars + accessories (presumably by copying items from the Library to their inventory), which weren’t removed as a part of the “recall”.
  • The appearance of the bodies + accessories also sparked a fair degree of forum discussion, approximately starting towards the bottom of page 17 of this thread.
  • In reference to the thread, LL encourage those who did manage to retain the Senra bodies and are observing issues / have concerns to continue to record feedback there, as “all eyes” involved in the project are watching that thread.

glTF Materials and Reflection Probes

Project Summary

  • To provide support for PBR materials using the core glTF 2.0 specification Section 3.9 and using mikkTSpace tangents, including the ability to have PBR Materials assets which can be applied to surfaces and also traded / sold.
  • There is a general introduction / overview / guide to authoring PBR Materials available via the Second Life Wiki.
  • For a list of tools and libraries that support GLTF, see https://github.khronos.org/glTF-Project-Explorer/
  • Substance Painter is also used as a guiding principal for how PBR materials should look in Second Life.
  • Up to four texture maps are supported for PBR Materials: the base colour (which includes the alpha); normal; metallic / roughness; and emissive, each with independent scaling.
  • Given the additional texture load, work has been put into improving texture handling within the PBR viewer.
  • In the near-term, glTF materials assets are materials scenes that don’t have any nodes / geometry, they only have the materials array, and there is only one material in that array.
  • As a part of this work, PBR Materials will see the introduction of reflection probes which can be used to generate reflections (via cubemaps) on in-world surfaces. These will be a mix of automatically-place and manually place probes (with the ability to move either).
  • The overall goal is to provide as much support for the glTF 2.0 specification as possible.
  • As a result of the updates, SL’s ambient lighting will change (e.g. indoor spaces will appear darker, regardless of whether or not Shadow are enabled), and so there will be a period of adjustment for users (e.g. opting to install lighting in indoor spaces, choosing between the HDR lighting model of glTF or opting to set a sky ambient level).
  • The viewer is available via the Alternate Viewers page.
  • The simulator code is now more widely available on the Main Grid, including some sandbox environments, but still in RC. Demonstration regions might be found at: Rumpus Room, Rumpus Room 2, Rumpus Room 3, Rumpus Room 4, Rumpus Room 5.
  • Please also see previous CCUG meeting summaries for further background on this project.

Status

  • Many in the team have been out-of-office recently, and so work had slowed for a while.
  • The focus remains on bug fixing within both the viewer and the simulator code.

Double-Sides Materials Concerns

A request was made for LL to remove double-sided materials from the PBR work, due to the following concerns:

  • Inexperienced creators misunderstanding the capability (e.g. a content creator who makes a pendant with 500,000 triangles and applying materials to all of them, overriding backface culling), and:
  • Clothing creators who add additional tris along the edges of clothing (e.g. cuffs, lapels, collars, etc.), to given the illusion of an “inner” material instead utilising double-sided materials instead, leading to:
  • The potential of both of these leading to noticeable viewer performance impacts (e.g. due to doubling the amount of rasterising the viewer must perform).

In response Runitai Linden noted:

  •  Double-sided materials is a part of the glTF specification, and so will remain  within the PBR project, and so forms a part of the overall requirements for obtaining Khronos 3D commerce certification, which LL would like to achieve for Second Life, and for that reason will not be removed.
  • In terms of performance LL believe:
    • Double-sided materials generally do not get rasterised twice (e.g. if you are looking at the front face of a leaf with double-sided materials, the back face is not rasterised) – although there are some exceptions to this.
    • Double-sided materials are a “fill” hit, not a per triangle hit, so the performance hit decreases exponentially as the camera moves away from the object – so for actual double-sided objects, it is a performance win.
  • To help safeguard against accidental misuse, the option to apply double-sided materials must be explicitly enabled when uploading, even if the materials themselves have been created as double-sided (if the option is not explicitly set, then they will be uploaded as single-sided).
  •  The issue does admittedly have edge-cases, and there are issues around any implementations for double-sided materials (e.g. how do you penalise for incorrect use? Increased LI? But then – a) what about worn items (which are immune to LI), and b) how does the viewer differentiate between “correct” use of double-sided materials and an “incorrect” use, in order to avoid penalise good practices in error?
  • However, LL are not going to disable / artificially limit the use of double-sided materials due to the potential for misuse, either accidental or deliberate.

PBR Mirrors

  • This is a follow-on project to the PBR Materials, intended to provide a controlled method to enable planar mirrors in SL (i.e. flat surface mirrors which can reflect what is immediately around them, including avatars).
  • As per my previous CCUG update, the approach being taken is to use a “hero probe”.
    • This uses a materials flag added to a surface which allows it to be considered as a mirror face, based on the proximity of a camera to it.
    • When a camera is within the expected range, the flag will instruct the viewer to create a “hero probe”, rendering high resolution (512×512) reflections on the mirror surface until such time as the camera moves away.
    • It is an approach which allows for multiple mirrors within a scene, whilst minimising the performance impact to only one mirror per viewer.
  • The concept is now working in tests, and depending on performance, it is possible the viewer might be allowed to support up to two hero probes at a time: one for any nearby mirror surface, and the other for generating reflections on any nearby Linden Water.
  • It is hoped that a project viewer will be available for public testing of the idea will be available Soon™.

ARC  – Avatar Render Cost

  • Intended to be a means of calculating the overall cost of rendering individual avatars by the viewer, ARC has long been acknowledged as inaccurate.
  • Currently, the project to adjust both ARC calculations and the actual cost of rendering in-world objects to make them more reasonable – Project ARCTan – remains inactive.
  • The problem with such metrics like ARC is that they depend on a range of analyses which, when combined, do not necessarily result in an accurate reflection of real-world rendering very well.
  • However, those curious about the rendering cost can use:
    • World→Improve Graphic Speed→Your Avatar Complexity to seen the render impact (in ms, currently for the CPU, but with the PBR viewer, for the GPU) of their own attachments can have on own and other viewers.
    • World→Improve Graphic Speed→Avatars Nearby to see the rendering impact of other avatars within view.
    • Note that both will fluctuate do to the general “noise” of rendering, however, the generated figures are far more accurate in real terms than those for ARC.
    • Details of these capabilities – first deployed in Firestorm, and contributed to Linden Lab for inclusion in all viewers, can be found in this blog, here.
  • Questions were asked over the ability to see these figures displayed over avatars heads vs. having to go to a “specialised” menu, with some at the meeting pointing to the overhead display being preferable, because it it “there”. However, this overlooks the facts:
    • It could be received as “cluttering” the in-world view and reducing immersiveness.
    • If displayed as hover text, it could be easily disabled either by an dedicated UI setting or simply by exposing the debug to disable avatar-related hover text
    • Most particularly, any such display (even if added to name tags) would in fact adversely impact performance due the CPU / GPU cycles taken up by performing the calculations and then displaying them – with Runtai noting it can takes “several times longer” CPU time to calculate and display avatar render cost on the than it does to render the avatar.
  • The above led to a broader discussion on how to encourage better awareness of avatar impact on viewer performance (ARC shaming not being a positive approach to things), such as general education among users and having some form of “try before you buy” capability (if this were possible to implement) which would offer the ability to see the impact of wearing a specific attachment ahead of wearing it), or some form of inspection capability at upload which might encourage creators to go back and better optimise their avatar attachments.
    • One noted issue here is ensuring both sides of the equation have the tools to make more informed decisions: creators in terms of making their content more performant / efficient, and consumers to enable them to be able to better identify performant / efficient content. The latter is particularly important in its ability to drive market forces through users being able to naturally gravitate towards more efficient content.

Tags for Wearables

This was an idea mooted by the Lab in the meeting – not a project currently being worked upon.

  • A tag system which allows items with a certain tag to automatically replace another of the same tag type with a single click and without also replacing other items using the sane attachment point. For example, an item tagged as “hair” replaces the currently worn hair with a single click, but without also knocking off a hat also worn on the skull.
  • This was expanded upon by the idea of tags being used with demo items – the tag being used to perform tasks such as:
    • Only allowing the demo item to be worn within a certain location (e.g. the “dressing” area of the store).
    • Somehow records the item being worn prior to using the demo, o that it is automatically replaced when the demo item is removed.
  • The problem with the latter idea is that everyone uses demos differently, so assigning a single place at which a demo can be tried is a non-starter (do we really wany people trying demos at already busy events? What about items purchased via the MP or affiliate vendors, what location should be assigned to them? How is the creator to differentiate? Multiple versions of the same item for different points-of-sale? What about people who don’t have a home location by use sandboxes, but the demo tagged for use only within the avatar’s home location? Can this realistically even be done?).
  • An alternative suggestion for tags put forwards at the meeting was to have them as a part of the upload process, so creators could be reminded / encouraged to specific the desired attachment point via a tag list, so that users are not left with items defaulting to their avatar’s right hand.
  • There are a range of issues over any tag system, including:
    • a) How well the option would be used unless enforced; b) Even if enforced, how many content creators, would actually define the preferred attach point over just selecting the first one on the list?
    • The idea leans towards WEAR, rather than ADD – so will not necessarily overcome the confusion of new users who wish to ADD an item to their avatar, only to find it knocks something else off of their avatar.
    • How many tags should be in the system? “Hair”, “shirt”, “pants”, “gloves”, “shoes” are all straightforward – but what about shawls or shoulder wraps? should they be classified as a shirt or a collar, or have their own tag or individual tags? How are rings, earrings, pendants, etc., be classified / tagged?

Next Meeting

† The header images included in these summaries are not intended to represent anything discussed at the meetings; they are simply here to avoid a repeated image of a gathering of people every week. They are taken from my list of region visits, with a link to the post for those interested.

2023 SL Puppetry project week #28 summary

Posted on by Inara Pey
Puppetry demonstration via Linden Lab – see below.  Demos video with the LL comment “We have some basic things working with a webcam and Second Life but there’s more to do before it’s as animated as we want.”

The following notes have been taken from chat logs and audio recording of the Thursday, July 13th, 2023 Puppetry Project meetings. Notes in these summaries are not intended to be a full transcript of every meeting, but to highlight project progress / major topics of discussion.

Project Overview

General Project Description as Originally Conceived

LL’s renewed interest in puppetry was primarily instigated by Philip joining LL as official advisor, and so it really was about streaming mocap. That is what Philip was interested in and why we started looking at it again. However since Puppetry’s announcement what I’ve been hearing from many SL Residents is: what they really want from “puppetry” is more physicality of the avatar in-world: picking up objects, holding hands, higher fidelity collisions. 
As a result, that is what I’ve been contemplating: how to improve the control and physicality of the avatar. Can that be the new improved direction of the Puppetry project? How to do it?

– Leviathan Linden

  • The project is rooted in the in idea of “avatar expressiveness”, referenced in a February 2022 Lab Gab session with Philip Rosedale and Brad Oberwager and officially introduced as Puppetry in August 2022 to provide a means by which avatars can mimic physical world actions by their owners (e.g. head, hand, arm movements) through tools such as a webcam and using technologies like inverse kinematics (IK) and the  LLSD Event API Plug-in (LEAP) system.
  • Since that time the project has expanded in size, attempting to encompass improving SL’s (somewhat primitive) IK system; investigating and developing ideas for direct avatar-to-avatar, avatar-to-object interactions (“picking up” an apple; high-fives. etc.); providing enhanced LSL integration for animation control; broader hardware support; adoption of better animation standards, etc.
  • This has led to a change in approach for the project – see below for more.

Bugs, Feature Requests and Code Submissions

  • For those experimenting with Puppetry, Jiras (bug reports / fixes or feature requests) should be filed with “[Puppetry]” at the start of the Jira title.
  • There is also a public facing Kanban board with public issues.
  • Those wishing to submit code (plug-ins or other) or who wish to offer a specific feature that might be used with Puppetry should:

Resources

Change In Approach

  • The Puppetry User Group meetings have, until now, been held on Aditi (the Beta grid) at the Castelet Puppetry Theatre, commencing at 13:00 SLT, and generally on alternate Thursdays to the Content Creation meetings, and as per the Second Life Public Calendar.
  • As of the July 13th, 2023 these meetings have now been suspended until further notice.
  • This does not mean the project is being abandoned; it was noted during the meeting that as several of those involved in the project attend other User Group (SUG) meetings – notably, but not exclusively, the Tuesday Simulator User Group meeting -, discussions on Puppetry can continue within hose meetings.
  • Explaining the decision, Simon Linden Noted:
There’s definitely a lot of interested tech and possible features with [Puppetry]. [However] the original idea of doing real-time mocap on webcams was like opening Pandora’s box in terms of features and ideas, and also was a lot harder than we expected … in the end I think it’s better to work on some fundamental tech that can be used in a lot of other ways – like IK, streaming, figuring out how animation data can work with scripts, solving some challenges like just doing a decent hand-shake.

– Simon Linden, July 13th, 2023

  • Elements of Puppetry which have thus far been confirmed as continuing as WIP projects comprised (but are not necessarily limited to):
    • The real time animation streaming component of Puppetry (forming something of a hybrid between the LEAP <-> viewer work already undertaken, and Leviathan Linden’s work in streaming animation playback from one viewer, through the simulator and to other viewers without any direct interaction with the animations by the simulator).
    • IK Improvements and updates (see below).
    • Improved animation import support (see below).
  • There are also broader discussions going on in the Lab regarding possible further overhaul of the animation system.
  • The above decision re: Puppetry meetings being the case, this will be the last of these my dedicated Puppetry Puppetry summaries for the time being, but I will continue to report on Puppetry / related work as and when it is discussed at other meetings such as SUG meetings and Content Creation User Group meetings.

Meeting Notes

Animation Import

  • One of the Puppetry expansions, improving / broadening animation import into Second Life was spun-off into its own project and June.
  • Notably with this work, LL is using Assimp (github: https://github.com/assimp/assimp), which supports some 30-40 animation formats (including the likes of .FBX and glTF), converting them to its own format for ease of import to multiple platforms.
  • The work is now in its own branch of the official viewer (DRTVWR-584, not currently ready for public consumption in any way).
  • This viewer uses the Assimp engine to read animation files, and the viewer extracts data from there for preview and then uploading as a SL animation. Animation imports it supports include:
    • BVH files, as per the current animation import within the viewer.
    • FBX format files.
    • Animations saved with the Mixamo skeleton, as supported by other tools.
  • The Mixamo element of the viewer is currently incomplete, but there is a focus on getting it wrapped up so the viewer can enter the project viewer pipeline at some point for public testing. However, when complete, it is hoped that importing an animation with a Mixamo skeleton from the likes of  Blender or a tool like Rokoko Studio should work fairly seamlessly.
  • To help with imports, Aura Linden has included an option on import to scale motion, which might be further automated slightly, for improved ease-of-use among less experienced content creators.
    • If this process is automated, it will include an capability for manual override of course for those who are more experienced with animation creation and import.

Inverse Kinetics  (IK) Updates

  • Leviathan Linden’s IK work has pretty much become a mini-project in its own right.
  • Most recently, he has been focused on implementing Forward And Backward Reaching Inverse Kinematics (FABRIK) – which is the fundamental algorithm for suggesting new joint positions in a range of applications, including 3D modelling.
  • This work has been in part a matter of trial-and-error, and most recently, Leviathan has been fixing issues of where constraints are enforced in FABRIK which impact the SL avatar, although he still has some more constraints to fix.
  • Fixing these issues has required additional visualisation / debugging tools, which he’s having to code for himself.

Additional Notes

  • A further request was made for updating the bento reference skeletons on the wiki, which are reported as being “broken”, per BUG-10981 “Test content for public Project Bento Testing wiki page” and this Content Creation Discord channel discussion. This will be chased internally at LL to see if action is being taken.

2023 week #27: SL CCUG + TPVD meetings summary

Posted on by Inara Pey

[REN] May, 2023 – blog post
The following notes were taken from my audio recording and chat log transcript of the Content Creation User Group (CCUG) meeting held on Thursday, July 6th, and the Third Party Viewer Developer (TPVD) meeting held on Friday, July 7th, 2023. 

Meetings

  • The CCUG meeting is for discussion of work related to content creation in Second Life, including current work, upcoming work, and requests or comments from the community, together with viewer development work.
  • The TPV Developer meeting provides an opportunity for discussion about the development of, and features for, the Second Life viewer, and for Linden Lab viewer developers and third-party viewer (TPV) / open-source code contributors developers to discuss general viewer development.
  • As a rule, both meetings are:
    • Held in-world and chaired by Vir Linden.
    • Conducted in a mix of voice and text.
    • Held at 13:00 SLT on their respective days.
    • Are subject to the schedule set within the SL Public Calendar, which includes locations for both meetings (also included at the end of these reports).
    • Open to all with an interest in content creation / viewer development.
  • As these meetings occasionally fall “back-to-back” on certain weeks, and often cover some of the same ground, their summaries are sometime combined into a single report (as is the case here). They are drawn from a mix of my own audio recordings of the meeting + chat log (CCUG), and from the video of the TPVD meeting produced by Pantera Północy (which is embedded at the end of the summaries for reference) + chat log. Not that they are summaries, and not intended to be transcripts of everything said during either meeting.

Viewer News

  • The Windows 32 + macOS pre-10.13 RC, version  6.6.13.580794 was rapidly promoted to de facto release status on July 6.
  • The Second Life Project Inventory Extensions viewer updated to version 6.6.13.580656, on July 6.

The remaining viewers in the pipeline stand as:

General Viewer Notes:

  • The Windows 32 + macOS pre-10.13 viewer is the last official viewer supporting either the Win 32-bit version or Mac OS versions prior to version 11. As viewers for 64-bit only and MacOS 11+ become the defacto releases, this viewer (as it is and without further update) will be moved to a side cohort, and used when someone running a pre-11 MacOS release or a Windows viewer incapable of running the 64-bit version of the viewer will receive this version instead.
  • Maintenance T has returned to being the most likely RC viewer due for promotion to de facto release status, now that its high crash rate has (hopefully) been brought under control.
  • The Emoji project viewer is liable to see further improvements to the Emoji picker in the UI.

glTF Materials and Reflection Probes

Project Summary

  • To provide support for PBR materials using the core glTF 2.0 specification Section 3.9 and using mikkTSpace tangents, including the ability to have PBR Materials assets which can be applied to surfaces and also traded / sold.
  • There is a general introduction / overview / guide to authoring PBR Materials available via the Second Life Wiki.
  • For a list of tools and libraries that support GLTF, see https://github.khronos.org/glTF-Project-Explorer/
  • Substance Painter is also used as a guiding principal for how PBR materials should look in Second Life.
  • Up to four texture maps are supported for PBR Materials: the base colour (which includes the alpha); normal; metallic / roughness; and emissive, each with independent scaling.
  • Given the additional texture load, work has been put into improving texture handling within the PBR viewer.
  • In the near-term, glTF materials assets are materials scenes that don’t have any nodes / geometry, they only have the materials array, and there is only one material in that array.
  • As a part of this work, PBR Materials will see the introduction of reflection probes which can be used to generate reflections (via cubemaps) on in-world surfaces. These will be a mix of automatically-place and manually place probes (with the ability to move either).
  • The overall goal is to provide as much support for the glTF 2.0 specification as possible.
  • The viewer is available via the Alternate Viewers page.
  • Please also see previous CCUG meeting summaries for further background on this project.

Status

  • The simulator code is now more widely available on the Main Grid, including some sandbox environments, but still in RC. These regions comprise:
  •  However, it should be noted that:
    • Bug-fixing within the simulator code is also on-going, so those testing PBR materials where the support is available should take note of this.
    • Obviously, to take advantage of these regions (both to upload PBR Materials and to see the new PBR environment lighting + the demonstration objects provided within some the regions, you must be running the PBR Materials RC viewer.
    • The cost of upload for Materials without any textures is free; however, the L$10 texture upload fee is charged on a per texture basis for any included within a PBR material surface (up to four can be included).
    • Any PBR materials content created within these environments which is later rezzed in any region that is not Materials-enabled, will become “material-less” in a non-recoverable way, and will need to be recreated.
    • Because of the above point, until PBR support is fully gird-wide, any attempts to put PBR-enabled goods on the Marketplace will be sanctioned.
  • General bug fixing on the viewer is also continuing, and the spread PBR across the grid is largely down to a mix of bug fixing and getting everything stable, and seeing if any particularly nasty blockers or bug rear up.
  • As an aside, the additional texture overhead for PBR Materials has been one of the drivers behind increasing the amount of VRAM the viewer can use for textures, as also now found within the PBR Materials viewer.

PBR Mirrors

  • This is a follow-on project to the PBR Materials, intended to provide a controlled method to enable planar mirrors in SL (i.e. flat surface mirrors which can reflect what is immediately around them, including avatars).
  • The approach Geenz Linden is taking towards enabling mirrors is to use a “hero probe” concept. In short:
    • With PBR, reflections are generated using reflection probes, per the notes above. These are capped at a maximum resolution of 128×128 per object face; hero probes will be capped at 512×512 per object face, providing a mush higher resolution of reflection.
    • Hero probes are an automated selection process. “Standard” probes cannot be converted to hero probes by manual intervention, nor can they be created.
    • A hero probe is selected on the basis of the “mirror” surface / probe and the proximity of the viewing avatar / camera to it.
    • Thus, if there are 5 mirrors place in a room, only the mirror closest to any given avatar / avatar camera will be used as a hero probe within that avatar’s viewer; the other four will remain “standard” probes, until such time as the avatar / camera moves closer to one of them, at which point it will become the hero probe, and the former hero will revert to a standard probe.
  • The primary use case for this work is mirrors, but it is possible, depending on performance impact, etc., that in the future, one additional hero probe might be allowed per scene, and the use case broadened (e.g. the water reflections).

In Brief

Via the TPVD meeting

  • Simulator updates: two simulator RC are currently with QA:
    • One includes updates to the avatar arrival code (i.e. handling avatars being received into the region via TP) which should help reduce the “viewer freeze” / frame rate drop others in the region can experience.
    • The other includes bug fixes and additional LSL functions.
  • General discussions on:
    • Anecdotal spotting of increased teleport failures, but lacking specifics on location, repro, etc. LL’s gridwide stats on TPs is not reflecting any noticeable uptick in failures. Therefore, those who do encounter them frequently are asked to:
      • Log where / when / how such failure occur.
      • If a pattern emerges (e.g. failures consistently occur when teleporting after doing X, or, say, in the first TP are logging-in, or anything else which suggests a possible underlying pattern), to file a Jira bug report, providing as much of the info they’ve gathered as possible.
    • A similar anecdotal report on people seeing themselves (or others as clouds for extended periods with in increased frequency. Again, where this is seen to happen consistently, a bug report is requested.
    • The official viewer CHUI + lack of chat bar (which also leans into what appears to be a lack of awareness that the Chat Console is not a “Firestorm feature”, but something generally available to people – although new users may need their attention directed to (and how to open chat from it), as the fade-out can case thigs to be missed and the click-to-open chat is not obvious.
    • On options available if TPVs that are not available in the official viewer (or possibly, with some, just not exposed through the UI), but which possibly should be.
    • Please refer to the TPVD meeting video below for further details on the above discussions.

Next Meetings

† The header images included in these summaries are not intended to represent anything discussed at the meetings; they are simply here to avoid a repeated image of a gathering of people every week. They are taken from my list of region visits, with a link to the post for those interested.

2023 week #26: SL CCUG meeting summary

Posted on by Inara Pey

Pemberley – May 2023, blog post
The following notes were taken from my audio recording and chat log transcript of the Content Creation User Group (CCUG) meeting held on Thursday, June 15th, 2023 at 13:00 SLT. 

These meetings are:

  • For discussion of work related to content creation in Second Life, including current work, upcoming work, and requests or comments from the community, together with viewer development work.
  • Usually chaired by Vir Linden, and dates and times can be obtained from the SL Public Calendar.
  • Conducted in mixed voice and text chat. Participants can use either when making comments  or ask or respond to comments, but note that you will need Voice to be enabled to hear responses and comments from the Linden reps and other using it. If you have issues with hearing or following the voice discussions, please inform the Lindens at the meeting.

The following is a summary of the key topics discussed in the meeting, and is not intended to be a full transcript of all points raised.

Viewer News

  • Maintenance T RC viewer updated to version 6.6.13.580700 on Wednesday, June 28th.
  • The GLTF viewer updated to version 7.0.0.580717 on Tuesday, June 27th.

The rest of the official viewers currently in the pipeline remain as:

  • Release viewer: Maintenance S RC viewer, version 6.6.12.579987, dated May 11, promoted May 16.
  • Project viewers:

In addition:

  • Maintenance T is likely the next RC viewer remains the next in line for promotion to de facto release status.
  • The Emoji project viewer is liable to see further improvements to the Emoji picker in the UI.

glTF Materials and Reflection Probes

Project Summary

  • To provide support for PBR materials using the core glTF 2.0 specification Section 3.9 and using mikkTSpace tangents, including the ability to have PBR Materials assets which can be applied to surfaces and also traded / sold.
  • There is a general introduction / overview / guide to authoring PBR Materials available via the Second Life Wiki.
  • For a list of tools and libraries that support GLTF, see https://github.khronos.org/glTF-Project-Explorer/
  • Substance Painter is also used as a guiding principal for how PBR materials should look in Second Life.
  • Up to four texture maps are supported for PBR Materials: the base colour (which includes the alpha); normal; metallic / roughness; and emissive, each with independent scaling.
  • Given the additional texture load, work has been put into improving texture handling within the PBR viewer.
  • In the near-term, glTF materials assets are materials scenes that don’t have any nodes / geometry, they only have the materials array, and there is only one material in that array.
  • The overall goal is to provide as much support for the glTF 2.0 specification as possible.
  • To provide support for reflection probes and cubemap reflections.
  • The viewer is available via the Alternate Viewers page.
  • Please also see previous CCUG meeting summaries for further background on this project.

Status

  • The new RC viewer includes the automatic adjustment to make skies “PBR-ified” (so the checkbox in Graphics Preferences to disable automatically apply reflection probe ambience on skies that do not have it set, so they will be set to a default of 1). Probe ambience must now be set to 0 through the sky settings to deactivate HDR rendering (so no dynamic exposure so the sky is not brightened, and objects with PBR materials will appear duller than intended), and tone mapping is turned off.
    • However, there is a recognised bug related to probe ambience (BUG-234060 “[PBR] Simulator Clamping Environment’s Reflection Probe Ambience to 1”, and it is hoped a fix for this will be in the next simulator build – although it may take a while for the build to make its way into a deployment.
  • The-user supplied PBR content created on Aditi is being transferred to the PBR regions on the Main grid in readiness for more widespread viewing / testing.
  • The LDPW will be producing a PBR demonstrator which will be available through the viewer library.
  • The simulator-side support for glTF Materials is due to be expanded with further simulator RC deployments, which will notably include some sandbox environments.
  • However, it should be noted that:
    • Any glTF / PBR materials content created within these environments which is taken and then rezzed in any region that is not Materials-enabled, will become “material-less” in a non-recoverable way, and will need to be recreated.
    • Because of the above point, until the glTF support is fully gird-wide, any attempts to put PBR-enabled goods on the Marketplace will be sanctioned.
  • General bug fixing on both the simulator and the viewer is continuing.

PBR Terrain

Materials applied to Second Life terrains. Credit: Linden Lab
  • Per past meeting notes, Cosmic Linden is prototyping the application of PBR materials on terrain (see this blog post for more).
  • Further work on bug fixing, including correcting an issue with triplanar mapping to the for terrain repeats, particularly on steep elevation changes (so as to avoid the all-too familiar “stretching” seen with textures).
  • Important notes with this work:
    • It is not terrain painting. It is the application of PBR materials – terrain painting is described as “something that’s on the radar” at LL.
    • The work does not include support for displacement maps.
    • The work is currently only viewer-side, with no corresponding server-side support, the idea here being to prototype what might be achieved and testing approaches / results.

Level of Detail (LOD) Discussion

  • Level of detail (LOD) refers to the complexity of a 3D model representation. In short the idea is to reduce the load on the rendering system by reducing the complexity of the 3D object based on various of criteria (e.g. the distance of the object from the viewer / camera) and using various techniques.
  • Second Life uses the approach of Discrete Levels of Detail (DLOD) method – the use of discrete versions of the original with decreased levels of geometric detail to replace the more complex models using an algorithm primarily based on distance. This has both positives and negatives, some rooted in poor modelling practices by some content creators, others are inherent to flaws within SL itself.
  • In looking at glTF and mesh imports in the future,  LL is considering moving towards more automated and better optimised means of creating LODs to try to reduce some of the current issues.
  • One idea under consideration to achieve this is to leverage Simplygon (or see the Wikipedia entry), which although a proprietary tool, is available as a plug-in for a number of 3D modelling towards (including Blender); the idea being the glTF importer consume whatever output is generated by Simplygon at the creator’s end of the workflow (it being noted that “simply” integrating Simplygon into SL isn’t feasible).
    • Such an approach would offer advantages of optimisation whilst leaving the current upload process in place for those wishing to continue to use it, together with the continued ability to manually create LODs.
  • Some concerns over this possible approach raised at the meeting were:
    • It does not address issues of avatar complexity, which is potentially the biggest viewer performance hit – although in fairness, avatar complexity is really an issue requiring its own focus / project.
    • The nature of some of the Simplygon licensing statements (concern about which might be a mix of the genuine and a basic misunderstanding of legal terms used with regards to “how the Internet works”, so to speak; an issue we had back when LL changed some of the terminology within the SL TOS in 2013).
  • While there are other optimisations tools available, given the state of flux with things within the open source environment, and given Simplygon is recognised as an “industry standard”, offering a means to take output from in and bring it into Second Life is currently seen as the best strategy by the Lab, subject to the concerns raised about the licensing requirements.

In Brief

  • There have been reports of Animesh objects changing / becoming stuck in their lowest LOD in various circumstances, including on region crossing (see BUG-233691 “Animesh re-renders at lowest LOD for extended interval after long-range llSetRegionPos” + listed duplicates). This is believed to be a bounding box issue, and a fix is being developed which should hopefully make it into the next update of the Maint T RC viewer.
  • For PBR, the questions was raised about converting older content with the current Blinn-Phong materials to PBR where the base colour map is not available. The recommendation is that the two are kept separate: if the base colour map is not available for conversion, that Blinn-Phong should continue to be used. No attempt on LL’s part will be made to try to combine the two (PBR / Blinn-Phong) through any kind of conversion tools, as the two are currently entirely separate, allowing creators / users to apply PBR to items which may already have Blinn-Phong, and if the results aren’t good, strip the PBR away and immediately revert to the Blinn-Phong materials. Any attempt to allow “blending” between the two approaches will immediately break this.
  • One consideration actively being worked on within the Lab is glTF licenses and Second Life being able to recognise these. Licences tend to be built-in to glTF complaint 3D models, so as the end goal of the glTF project is to effectively be able to take a glTF model from (say) Sketchfab and drop it into SL, this should be done in respect of the object’s licensing (e.g. if it has a Creative Commons Non-Commercial license, it can be imported into SL but not re-sold after the fact or placed on the SL Marketplace).

Next Meeting

  • Thursday, July 6th, 2023.

† The header images included in these summaries are not intended to represent anything discussed at the meetings; they are simply here to avoid a repeated image of a gathering of people every week. They are taken from my list of region visits, with a link to the post for those interested.

2023 week #24: SL CCUG meeting summary

Posted on by Inara Pey
The Last Aokigahara Souls, April 2023 – blog post †

The following notes were taken from my audio recording and chat log transcript of the Content Creation User Group (CCUG) meeting held on Thursday, June 15th, 2023 at 13:00 SLT. 

These meetings are:

  • For discussion of work related to content creation in Second Life, including current work, upcoming work, and requests or comments from the community, together with viewer development work.
  • Usually chaired by Vir Linden, and dates and times can be obtained from the SL Public Calendar.
  • Conducted in mixed voice and text chat. Participants can use either when making comments  or ask or respond to comments, but note that you will need Voice to be enabled to hear responses and comments from the Linden reps and other using it. If you have issues with hearing or following the voice discussions, please inform the Lindens at the meeting.

The following is a summary of the key topics discussed in the meeting, and is not intended to be a full transcript of all points raised.

Viewer News

No official viewer updates up until the meeting, leaving the list as:

In addition:

  • Maintenance T is likely the next RC viewer in line for promotion to de facto release status; however, it currently has an elevated crashed rate compared to the current release viewer, so LL is looking to bring that down before any promotion in made.
  • The Emoji project viewer is liable to see further improvements to the Emoji picker in the UI.
  • All of the back-end work (simulator code and the inventory service) seen as necessary to support Inventory Thumbnails is now thought to be in place, opening the door for a Thumbnails project viewer “Soon™”, which is apparently awaiting QA clearance.
    • Apparently this now provides the option of displaying the contents of an inventory folder either as we’re currently familiar with them (icons and text) or in a “thumbnail view” – I’ll review this once the viewer is available.
    • It’s anticipated that as the feature goes mainstream, creators will include thumbnails with their product offerings, as well as users creating their own thumbnails of items already in their inventory.
  • A further Maintenance RC viewer (Maint U) is in development and could be surfacing “Soon™”.

glTF Materials and Reflection Probes

Project Summary

  • To provide support for PBR materials using the core glTF 2.0 specification Section 3.9 and using mikkTSpace tangents, including the ability to have PBR Materials assets which can be applied to surfaces and also traded / sold.
  • There is a general introduction / overview / guide to authoring PBR Materials available via the Second Life Wiki.
  • Substance Painter is also used as a guiding principal for how PBR materials should look in Second Life.
  • Up to four texture maps are supported for PBR Materials: the base colour (which includes the alpha); normal; metallic / roughness; and emissive, each with independent scaling.
  • Given the additional texture load, work has been put into improving texture handling within the PBR viewer.
  • In the near-term, glTF materials assets are materials scenes that don’t have any nodes / geometry, they only have the materials array, and there is only one material in that array.
  • The overall goal is to provide as much support for the glTF 2.0 specification as possible.
  • To provide support for reflection probes and cubemap reflections.
  • The viewer is available via the Alternate Viewers page.
  • Please also see previous CCUG meeting summaries for further background on this project.

Status

  • There is another viewer RC in the works. This has been delayed whilst a number of bugs were resolved, but it is now with QA and pending their OK, it should be surfacing in the very near future.
  • The water fog density range has been updated in the PBR viewer. It was capable of being set from -10 through to 10, but is now  “almost zero” through to 100, with the higher numbers indicating higher fog density (and lower visibility).
  • Sky settings:
    • As per my notes from the last CCUG, all “legacy” skies which do not have a probe ambience set, will have one automatically applied by the viewer.
    • This has lead to a checkbox being including in the Graphic Preferences to disable the automatic application. However, it is possible / likely the check box will be removed.
    • This will mean the only way to have “legacy” skies render as they did pre-PBR will be to set the probe ambience setting to zero in the sky settings.
    • Setting probe ambience to 0 effectively deactivates HDR rendering and tone mapping etc.
  • The above requirement to set the probe ambience to zero has two further impacts:
    • It will be the only means by which “legacy” materials can be rendered as they did pre-PBR; the viewer “hack” to desaturate and “nudge” the luminance of “legacy” materials to have them render as they did pre-PBR has been removed due to combination of factors, including a noticeable performance hit.
    • The hack that allowed full bright objects to be exempt from dynamic exposure (so they would always stay the same brightness no matter what the exposure) has also had to be removed from the viewer, again as a result of a noticeable performance hit (an extra texture sample for each pixel). So, full bright object will, in future versions of the PBR viewer, be subject to the same exposure rules as everything else in the scene.
      • This means that full bright objects will still appear to be lit by 100% of the light and be unaffected by things like local lights, but the overall dynamic exposure of the scene will also brighten / darken them in keeping with the rest of the scene, and tone mapping will be applied.
      • SL photographers who display their images in-world with full bright are going to need to experiment with using sky settings with tone mapping disabled when taking their shots in order to avoid it being applied twice to images (e.g. once when the snapshot was taken and again when displayed in-world) and having this unduly affect the full bright rendering of the image.
  • Screen Space Reflections (SSR) – (non-SL overview):
    • Glossy support led to performance hits, particularly when there were a lot of alpha objects in the scene being rendered.
    • This has now been mitigated via a number of means; e.g. SSR is not even applied if it exceeds a certain roughness value, nor is it applied to alpha blended objects (the “old fade-out” method is used – that is: the higher the roughness, the weaker the SSR and the greater the reliance on reflection probes).
    • There have also been some general adjustments to SSR to again try to reduce the performance hits, whilst also allowing more distant objects to be reflected off of surfaces – said to be noticeable on water and flat surfaces.
  • The SSR work provoked a reminder that it is important not to abuse the use of alpha surfaces when SSR is used in rendering.
    • For example:
      • If the object is just a single layer users will be looking through to see what lays beyond, than alpha blending “may be the right choice”.
      • However, alphas should not be used to create layered effects (e.g. setting the faces of a surface to transparent for a “window” and then applying a further alpha to give the window a “frosted glass” look, and having the viewer composite them when rendering. Instead, both alphas should be baked down into one material in PhotoShop (or similar) & applied).
    • This sparked a discussion on how general users who dabble in content creation and who may be used to working in certain ways, can be properly educated to understand things like this – which may be known and understood by mainstream content creators, but could easily pass others by, as there is no means within the viewer UI of telling what is “good” or “bad” techniques when banging things together (at least until frame rates disappear through the floor and into the cellar).
    • It was therefore suggested again that documentation on the wiki – “best practices” / a “bible” for PBR Materials creation / use really should accompany the formal release of PBR Materials, and this should be clearly pointed at though blog posts, etc.

PBR Terrain

Materials applied to Second Life terrains. Credit: Linden Lab
  • Per past meeting notes, Cosmic Linden is prototyping the application of PBR materials on terrain (see this blog post for more).
  • The focus currently is on adding triplanar mapping to the for terrain repeats, particularly on steep elevation changes (so as to avoid the all-too familiar “stretching” seen with textures).
  • The above does potentially incur a performance hit, as it is noted as being for “sufficiently capable machines”, so Cosmic has also been working on trying to optimise performance elsewhere in the use of materials on terrain, as well as carrying out some bug fixing.
  • Important notes with this work:
    • It is not terrain painting. It is the application of PBR materials – terrain painting is described as “something that’s on the radar” at LL.
    • The work does not include support for displacement maps.
    • The work is currently only viewer-side, with no corresponding server-side support, the idea here being to prototype what might be achieved and testing approaches / results.

In Brief

  • It was re-iterated that long-term graphics support on the MacOS side will be MoltenVK (with Vulkan the likely route for Windows), with Zink also being looked at.
  • New user retention:
    • It has been suggested that as many in the various SL communities – those running Community Gateways, content creators, etc., – all have a interest in the new-user experience and brining people into SL, that a semi-regular “New User Experience” meeting might be established by LL at which ideas can be more readily discussed, feedback given and communities and creators more directly engaged with LL’s own efforts (e.g. developing an ecosystem of clothing, etc., to support the (still) upcoming new starter avatars. etc.).
    • It was pointed out that one of the best ways to encourage retention is for existing users to be welcoming, polite and supportive of new users (clubs that have scripts auto-ejecting people just because they are only a handful of days old or on the basis that an avatar is not PIOF, for example, aren’t exactly rolling out the welcome mat, for example).
    • The above spun into a general discussion on content, content moderation, and similar.
  • There was a general discussion on Land Impact (LI) and how it is “too high” – although this is highly subjectively, given the diversity of content and its uses in SL (static, animated, Animesh, etc. – how high is “too high”? Is “too high” down to more a lack of understanding of actual rendering + simulator load costs than an objective measurement? etc.). That said, it was acknowledged that more could be done t help “improve” LI in some areas.
  • The above spun into a discussion on physics costs, and how convex hull physics forms are not the most efficient for SL due to the complexity of the tools used to create them, and the ease with which mistakes can be made in creating them, with indications that as the glTF project moves towards support for mesh uploads how physics shapes are used / applied is likely to be be revised.

Next Meeting

  • Thursday, June 29th, 2023.

† The header images included in these summaries are not intended to represent anything discussed at the meetings; they are simply here to avoid a repeated image of a gathering of people every week. They are taken from my list of region visits, with a link to the post for those interested.

2023 SL Puppetry project week #23 summary

Posted on by Inara Pey
Puppetry demonstration via Linden Lab – see below.  Demos video with the LL comment “We have some basic things working with a webcam and Second Life but there’s more to do before it’s as animated as we want.”

The following notes have been taken from chat logs and audio recording of the Thursday, June 8th, 2023 Puppetry Project meetings. Notes in these summaries are not intended to be a full transcript of every meeting, but to highlight project progress / major topics of discussion.

Meeting And Project Overview

  • The Puppetry User Group exists to provide an opportunity for discussion about the development of, and feature for, the upcoming Second Life Puppetry project(see below for details), bugs, and feature ideas.
  • These meetings are conducted (as a rule):
    • On Aditi (the Beta grid) at the Castelet Puppetry Theatre, commencing at 13:00 SLT.
    • Those encountering issues in logging-in to Aditi should contact Second Life support for assistance.
    • Generally on alternate Thursdays to the Content Creation meetings.
    • Comprise a mix of text and voice – attendees can use text only, if preferred, but should enable to hear comments / responses given in voice.
  • These meetings are open to anyone with a concern / interest in the Puppetry project, and form one of a series of regular / semi-regular User Group meetings conducted by Linden Lab. Dates and times of all current meetings can be found on the Second Life Public Calendar, and descriptions of meetings are defined on the SL wiki.

General Project Description as Originally Conceived

LL’s renewed interest in puppetry was primarily instigated by Philip joining LL as official advisor, and so it really was about streaming mocap. That is what Philip was interested in and why we started looking at it again. However since Puppetry’s announcement what I’ve been hearing from many SL Residents is: what they really want from “puppetry” is more physicality of the avatar in-world: picking up objects, holding hands, higher fidelity collisions. 
As a result, that is what I’ve been contemplating: how to improve the control and physicality of the avatar. Can that be the new improved direction of the Puppetry project? How to do it?

Leviathan Linden

  • Previously referred to as “avatar expressiveness”, Puppetry is intended to provide a means by which avatars can mimic physical world actions by their owners (e.g. head, hand, arm movements) through tools such as a webcam and using technologies like inverse kinematics (IK) and the  LLSD Event API Plug-in (LEAP) system.
    • Note that facial expressions and finger movements are not currently enabled.
    • Most movement is in the 2D plain (e.g., hand movements from side-to-side but not forward / back), due to limitations with things like depth of field tracking through a webcam, which has yet to be addressed.
  • The back-end support for the capability is only available on Aditi (the Beta grid) and within the following regions: Bunraku, Marionette, and Castelet.
  • Puppetry requires the use of a dedicated viewer, the Project Puppetry viewer, available through the official Second Life Alternate Viewers page.
  • No other special needs beyond the project viewer are required to “see” Puppetry animations. However, to use the capability to animate your own avatar and broadcast the results, requires additional work – refer to the links below.
  • There is a Puppetry Discord channel – those wishing to join it should contact members of LL’s puppetry team, e.g. Aura Linden, Simon Linden, Rider Linden, Leviathan Linden (not a full list of names at this time – my apologies to those involved whom I have missed).

Additional Work Not Originally In-Scope

  • Direct avatar / object / avatar-avatar interactions (“picking up” an apple; high-fives. etc.).
  • Animations streaming: allowing one viewer to run animations and have them sent via the simulator to all receiving viewers without any further processing of the animations by those viewers.
  • Enhanced LSL integration for animation control.
  • Adoption of better animation standards – possibly glTF.
  • Given the project is incorporating a lot of additional ideas, it is likely to evolve into a rolling development, with immediate targets for development / implementation decided as they are agreed upon, to be followed by future enhancements. As such, much of what goes into the meetings at present is general discussion and recommendations for consideration, rather than confirmed lines o development.

Bugs, Feature Requests and Code Submissions

  • For those experimenting with Puppetry, Jiras (bug reports / fixes or feature requests) should be filed with “[Puppetry]” at the start of the Jira title.
  • There is also a public facing Kanban board with public issues.
  • Those wishing to submit code (plug-ins or other) or who wish to offer a specific feature that might be used with Puppetry should:

Resources

Meeting Notes

  • The viewer remains at  version 6.6.12.579958, issued on Thursday, May 11th.
    • This update includes access to Rider Linden’s experimental attachment point tracking & forwarding to the server feature.
    • It also includes various incremental improvements to handling puppetry, such as support for parsing binary LEAP data from the LEAP script.
  • Work has slowed a little due to Linden staff being out-of-office recently (hence why no meeting since May 11th), and personnel on the Puppetry project also working on other simulator projects.
    • This has notably impacted Leviathan Linden’s work on Inverse Kinetics (IK), which has had a knock-on impact slowing Rider Linden’s work on LSL support for driving puppetry.
    • However, progress has resumed on the IK work and while described as currently “not stable”, and problems are still to be solved in situations where a target position is too far away for a joint in the skeleton to reach, or where multiple joints (e.g. 5 or 6) are involved.
    • One issue that is proving difficult to handle is the default avatar mesh joint weighting is incorrect along the forearm and wrist. What is required is two distinct joints at the forearm to do mesh bending correctly: a hinge at the elbow and also a twist constraint along the forearm bone, toward the wrist, rather than (as is currently the case) treating the wrist as a ball joint. This may be the subject of further internal discussion at LL as Leviathan gets more of the IK work nailed down.
  • WRT IK:
    • Leviathan is looking to solve the targeting issues first, then work back to ensure that there are no collisions between a limb and avatar body (e.g. reaching across the avatar’s body to pick something up, and the avatar’s elbow / part of the arm appears to go through the body).
    • Forward And Backward Reaching Inverse Kinematics (FABRIK) – which is the fundamental algorithm for suggesting new joint positions in a range of applications, including 3D modelling – is the route of choice of the Lab; however, adopting to FABRIK is taking some trial and error.

Additional Notes

  • Aura Linden is here but she is working on the Animation importer project which has been split off from the Puppetry project. Currently, the status is animations can be import import animations from some tools/formats, but others aren’t working yet.
    • It was noted at the last meeting, that for animation import, LL is looking towards using / supporting Assimp (github: https://github.com/assimp/assimp), which supports some 30-40 animation formats, converting them to it ownformat for ease of import to multiple platforms. Notably, it supports .FBX and glTF, so it fits with the Lab’s goal of utilising glTF for materials, mesh imports, etc.

Date of Next Meeting

  • Thursday, June 22nd, 2023, 13:00 SLT.