2022 CCUG meeting week #18 summary: reflection probes

The following notes were taken from my audio recording and chat log of the Content Creation User Group (CCUG) meeting held on Thursday, May 4th 2022 at 13:00 SLT. These meetings are chaired by Vir Linden, and their dates and times can be obtained from the SL Public Calendar.

This is a summary of the key topics discussed in the meeting and is not intended to be a full transcript.

Official Viewers Update

• On Wednesday, May 4th, the de facto release viewer was updated to version 6.5.5.571282, formerly the MFA RC viewer, dated April 26. See my notes on this viewer here.
• On Thursday, May 5th, the Performance Improvements RC viewer updated to version 6.6.0.571507.
  • The focus for this viewer is now bug and crash fixes, rather than adding functionality.
  • This update also sees the viewer merged with the release viewer MFA code base, and so will require anyone who uses MFA and who has not logged-in to Second Life using an MFA token to do so -follow the link above for details.

The rest of the official viewers remain as:

• Release channel cohorts:
  • Makgeolli Maintenance RC viewer (Maintenance M) viewer, version 6.5.5.570983, April 26.
• Project viewers:
  • Performance Floater project viewer, version 6.5.4.569531, March 18.
  • Mesh Optimizer project viewer, version 6.5.2.566858, dated January 5, issued after January 10.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

Graphics Update

• During the previous CCUG meeting, the potential direction for Physically Based Rendering (PBR) materials support was discussed at length, and concerns were raised that for PBR to be effective, SL’s lighting / reflections model would need to be updated.
• As a result, these branches of work have commenced:
  • The foundational work in creating a materials type with an associated inventory asset, as per the week #16 meeting. This  will initially comprise the ability to copy a texture entry (with its specific parameters) to inventory.
  • Foundational work on the PBR graphics pipe in the viewer.
  • Work on an implementation of reflection probes which can be used both with PDR shading and with legacy content. This formed the focus of this meeting.

Reflection Probes

• Runitai Linden envisages two forms of reflection probes:
  • Those that are part of a scene’s environment and are automatically placed, but which can (hopefully) be adjustable by scene builders – this is the current focus of the work, with the reflection probes married to the octree so they will align with octree nodes.
  • Reflection probes which can be specifically placed within structures by their creators (e.g. a probe within each room of a house) or which can be created and added to a legacy structure.
    • These would override “environment reflection probes” to prevent clashes (so that, for example, your nice shiny kitchen worktop is not reflecting the sky outside, but is offering reflections of the static objects on and around it.
    • Their size would be based on the volume they are attached to, which would allow for parallax correction, allowing the reflections to be accurately generated within a room space.

• These probes:
  • Comprise an object (prim) defined as a reflection probe which essentially uses the 360º snapshot code viewer to produce a cube map of its surroundings with each face set to 512×512 resolution (so 6 x 512×512), which can then be used to generate object “reflections” on surfaces within their volume of control.
    • This will require some additional checkboxes to be added to the build floater / viewer, but otherwise means probes can be inserted into “indoor” scenes using the existing build toolset.
  • Capture everything except avatars (so no reflections of your avatar walking past a shiny surface).
  • Initially trigger the generation of their cube maps by the viewer on entering a scene.
    • Maps are cached on disk, with up to 64 held in memory.
    • Currently only 8 maps are active at any one time. However, given sampling of just 8 maps for rendering can take up to 70% of an RTX 3080 GPU, this number is likely to be dropped to 4, unless a more targeted means of sampling for a given pixel can be determined.
  • Are updated whenever the number of objects in them changes or the Day Cycle changes, although the update rate is no more than one probe per frame.
  • Include Fresnel calculations when generating cube maps.

• The overall aim of this work is to provide a means to support more physically accurate reflections in SL than can be currently generated (seen as a requirement for PBR support). This work can then both:
  • Be integrated into the PBR work as that is added to Second Life.
  • Improve the existing SL environment map so that legacy content does not clash with PBR content as badly as might otherwise be the case – although it will not be a perfect blending of the two.
  • However, it will be “very, very hard for this to not break [some] content”, although the overall result should improve so SL looks.
  • Given this, the capability will need a lost of assistance with testing.
• Concern has been raised about how to limit / control the use of probes so as not to cause potential performance issues (e.g. by creators including a probe with all of their products that have a shiny / reflective surface).
  • The plan is to have every single pixel in a rendered scene covered by at least one probe via the automatic placement, so objects that are outside certainly should not need their own reflection probes.
  • Even so, it has been suggested that time is taken to produce a “best practices” guide to using reflection probes and to provide examples of bad probe set-ups.
• It was also suggested that a flag for “do not reflect” should be provided to help control what is included in a cube map in a given setting. It was not clear if this would be possible, so the question was set aside for the next meeting.
• General Notes:
  • This work is still in its early stages, and elements may change.
  • Several aspects of reflection generation have yet to be tackled (e.g. handling 100% transparent and 100% alpha textured surfaces).
  • The work has yet to be tied-into the existing graphics controls in the viewer so that everything works consistently and logically.
  • Some of the additional work required for generating reflections might be compensated for by optimising other elements of the shader system as this work progresses.
  • At some point this work will be available within a project viewer for testing on Aditi.

Note: while reflection probes formed the core of this meeting, this does not mean this work will be shipped ahead of the more PBR-related work. All three project branches are in their early stages, and so no decision on prioritisation among them has been taken.

PBR-Specific Work

• The PBR graphics pipe is still being plumbed-in.
• No work has really started with the PBR shader set.
• The plan is still to follow the glTF specification.
  • As this embraces the PBR Metallic Roughness workflow, this will be the first to be supported.
  • If demand surfaces, LL might add Specular Glossiness as a follow-up.

Next Meeting

• Thursday May 19th, 2022.