2022 CCUG + TPVD meetings week #11 summary: mirrors! (maybe)

The following notes were taken from:

• My audio recording and chat log of the Content Creation User Group (CCUG) meeting held on Thursday, March 17th 2022 at 13:00 SLT.
• My audio recording and the video recording by Pantera (embedded at the end of this piece, my thanks to her as always for recording the meetings) from the Third-Party Viewer Developer (TPVD) meeting on Friday, March 4th, 2022 at 13:00  SLT.

These meetings are chaired by Vir Linden, and their respective dates and times can be obtained from the SL Public Calendar.

This is a summary of the key topics discussed in each meeting and is not intended to be a full transcript of either. However, the video does provide a complete recording of the TPVD meeting, and timestamps to the relevant points within it are included in the notes below.

Available Viewers

[Video: 0:35-3:15 + notes from CCUG]

• The Performance Floater project viewer updated to version 6.5.4.569531, on March 18th. This viewer includes the Lab’s implementation of automatic adjustments to the viewer to try to maintain FPS. Monday, March 21st: Appears to have either been rolled back, or update to official viewers list was premature.

The list below reflects the rest of the currently available official Second Life viewers:

• Release viewer: version 6.5.3.568554 – formerly the Maintenance J&K RC viewer, promoted Monday, February 28 – No Change
• Release channel cohorts (please see my notes on manually installing RC viewer versions if you wish to install any release candidate(s) yourself).
  • MFA RC viewer, version 6.5.4.569309, issued on March 15.
  • Performance Improvements RC viewer version 6.6.0.569349, dated March 14.
  • Lao-Lao Maintenance RC viewer, version 6.5.4.569191, issued on March 11.
• Project viewers:
  • Mesh Optimizer project viewer, version 6.5.2.566858, dated January 5, issued after January 10.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

General Viewer Notes

• It is hoped that those using the official viewer and who are in the Performance Improvements RC cohort (or opt to manually install this viewer) will see noticeable performance improvements in terms of frame rates and fewer viewer stalls, when compared to previous SLV releases.
  • The RC has already seen some new bugs raised against it, and these are being worked on (e.g. BUG-231936).
  • It is still hoped this will be the next promotion to de facto release status, but this hinges on bug fixing and a decision on whether or not to release MFA as a dedicated viewer.
  • It is acknowledged that the work in this viewer and the Performance Floater project viewer needs to be reconciled with the work carried out by Beq Janus of the Firestorm team, whose code will be in the upcoming Firestorm 6.5.3 release.

MFA Viewer

• Summary:
  • Multi-factor authentication (MFA) is coming to the viewer.
  • As MFA is implemented in the official viewer, there will be a “grace” period to allow TPV adopt the viewer code.
  • During this period, users will be able to access SL on TPVs as they currently do now, regardless of whether or not they have opted-in to MFA.
  • After this “grace” period, all users who have opted in to MFA will be required to authenticate themselves when using any viewer to log-in to Second Life (with the usual 30-day period of valid authentication, as per secondlife.com MFA), but those who have not opted-in to MFA will see no difference in their log-in steps, regardless of whether the viewer they are using supports MFA.
• The RC version of the viewer, version 6.5.4.569309, was released on March 15th. This appears to have an authentication bug – BUG-231938 – related to how pass codes are parsed by the viewer.

• [CCUG meeting] The decision has yet to be taken on whether to maintain MFA as a dedicated viewer update or possibly merge it with another RC viewer (e.g. the Maintenance RC) to streamline the number of active viewer versions and the QA process.

Mirrors! (Maybe)

[Video: 4:05-16:29]

• In-world mirrors that reflect that reflect their surroundings – including avatars – have long been a requested capability in SL. Over the years, various attempts have been made to fake mirrors, such as using Linden Water and static photographic sets, though to “cheating” using projectors (albeit sans reflecting avatars) and the old means of duplicating sections of a build and inverting it so it appears to be reflections on a polished floor.
• In 2014, Zi Ree of the Firestorm team developed a means of generating real time reflections, including avatars, within the viewer, and produced a video on the idea. At the time, due to various reasons (most notably the impact on performance), LL decided mirrors could not be supported, which has tended to remain their position since.

• However, Mojo Linden (VP of Engineering) is now prepared to consider possibly enabling some implementation of mirrors in SL, although how this might be done is open to discussion, particularly as the potential for a massive performance impact is still very much a concern.
• Suggestions made by Mojo on how this might be done included:
  • To have user-initiated “mirrors” (that is, there are “mirror” objects in a space, but the rendering of their “reflections” is only triggered for a viewer based on something like their proximity to the object).
  • To possibly limit mirrors in terms of what they reflect (such is things that are directly in front of them, rather than much further in the background or only render avatars with the background left white.
• In terms of “triggering” mirrors, there was discussion on whether this should be automated or with user-controlled input:
  • Automated within the viewer:
    • The viewer detects the “mirror” object based on proximity, per above & renders the reflections. Discussion on this included the ideas that there could also be a Graphics preference option to completely enable or disable all mirror rendering by the viewer and a slider / drop-down to set the overall quality of the rendering of reflections (as is we already have for graphics quality (slider) and water reflections quality (drop-down)).
    • Automated via LSL (setting / unsetting flags on objects that trigger “reflection” rendering) or possibly using the interest list.
  • User controlled input: reflections are toggled in a manner similar to Media On A Prim: the user touches the “mirror” surface  and “reflections” are rendered only by their viewer – although this seems clunky / immersion breaking, even if it is an approach taken by games such as Cyberpunk 2077.
• One suggestion from those at the meeting was to take the Linden Water planar reflections (which are rendered anyway, whether the Linden Water is visible or not) and render them at a lower resolution in screen space.
  • Using the Water planar could help with things like reflective floors, mirror walls, etc., although screen space rendering in SL can be subject to clipping.
  • Screenspace reflections (SSR) have appeared in some TPVs – like Firestorm and Niran’s Viewer, but how well these work is open to question – for Firestorm, there were issues with the release of materials that resulted in the code being removed.
  • One  suggestion make for limiting performance impact was to limit the viewer to rendering only the nearest mirror surface and either use SSR for the remaining mirrors or, don’t render them at all
•  A lot depends on potential use-cases, and whether expectations can be managed – as there is a potential that any solution will not be able to meet all requirements, simply because of the risk of performance impact.
• Everything on mirrors is purely at the discussion stage right now and there is currently no formal project – and indeed, there might never actually be any project to implement mirrors; however, that the topic is being discussed by LL marks a significant shift in thinking.

In Brief

From the Content Creation Meeting

A lot of general discussion on a number of topics, none of which is currently the focus of any Lab project, including:

• Using physics to allow more “wiggling parts” – such as ears, etc., – on mesh avatars: a lot of this would come down to positioning the most suitable bones and rigging to them.
• Controlling / ordering joint position overrides (BUG-231904 / BUG-231579): this is currently handled by mesh UUID – which can be random from the POV of the user. However, trying to order using the order things are added to an avatar could lead to race conditions, as the outfit system doesn’t have any concept of prioritising attachments, therefore a more structured schema – such as adding a specific field that could be set for objects – but even this could have conflicts and also likely to be a non-trivial piece of work.
• Expansion of supported upload formats for mesh (e.g. glTF) and use of techniques such as PBR, with a stated preference (from creators) for SL handling of reflections / reflectivity to be overhauled before any attempt is made to add PBR support.
• Expanding Bakes on Mesh to support materials – which as noted over the last several meetings would require a significant update to the Bake Service, which LL is not planning on doing in the foreseeable future.
  • A further complexity here is managing the proper ordering of the additional maps. Example: some users may want their shirt partially blended with their bra / vest normal; others might want their shirt to cover their bra /vest normal map whilst others may not want their bra / vest to have a normal at all; so how can these three states be collectively handled in terms of layering?
• Further discussion on terrain – which is the subject of a project for 2022, although there is some split about increasing the texel density / increasing the resolution/repeats (and thus reducing the blurriness) + allowing normal maps, or allowing more in the way of mesh terrain (heightmaps, etc.).
  • Another request for terrain has been to allow greater compositing of textures (e.g. for more graduation in changes of textures by height or offering greater depth to terrain by layering-up textures.
  • A further request has been to allow terrain painting – e.g., being able to “paint in” dirt patches on grass, etc.
• A request for a scripted means to apply EEP settings to oneself beyond the inventory Apply to Self such that (as an example), you buy a pair of sunglasses or tinted goggles, and they apply EEP setting that make your surrounding look darker. This would require a means to read and apply the contents of an object,

From the TPVD Meeting

• [Video: 16:31-27:45 and 28:58-42:30] A broad-ranging discussion on the viewer build process and tool chain, moving to more recent graphics APIs (the likes of Vulkan and Metals are under consideration), resource utilisation within the Lab, future work on improving the viewer’s threading capabilities, etc.
  • Outside of the ongoing (/periodic) tool chain updates and graphics API investigations, none of discussion points are subject to any immediate / near-term project.
  • It was again pointed out that a issue in considering replacements for / alternatives to OpenGL is that a lot of users run older PC harder which is incapable of running more recent APIs like Vulkan.
  • As most of which will likely only be of interest to viewer devs, please refer to the video.
• [Video 42:44-43:28] ARCTan project and Legacy Profiles:
  • Initiated in 2020, ARCTan is an attempt to re-evaluate both avatar rendering costs and the cost of in-world scene rendering, with the initial focus on avatar rendering cost / impact, itself running on two tracks:
    • Developing a new UI element in the viewer pull together information from various menus / debugs to display useable information on avatars / attachments that are heavy in rendering cost, and what can be done.
    • Gathering data with the intent to revise and improve the actual formulas used for calculating avatar complexity, making them more relevant.
  • ARCTan effectively went into hibernation as a singular project in the latter part of 2021, and is currently awaiting development bandwidth.
  • [Video 45:09-46:05] One of the issues with ARCTan was being able to pull together the data in a fine enough detail to make any UI used to make changes worthwhile. Beq Janus has tackled a lot of this work for the upcoming Firestorm release, and this is liable to be fed into projects like ARCTan and the Lab’s own work in improving viewer performance and making options visible to users.
  • Legacy Profiles is a project to move avatar profile information back into the viewer and away from using web pages. There is project viewer (see the list at the top of this article), but it has been awaiting some back-end server work to be completed, and it is hoped that the project will be moving forward “Soon™”.
• [Video: 43:54-44:44] CEF: Chrome Embedded Framework (CEF) is the API used to manage media in SL. The SL version is now running several releases behind the current CEF version, and while a need to update has been noted, it is liable to require some significant work (e.g. updated UI elements as well as under-the-hood changes).
• [Video 46:05-end] miscellany text conversations, mostly on the topics noted above.