Tag: glTF

The following notes were taken from my audio recording and chat log of the Content Creation User Group (CCUG) meeting held on Thursday, June 16th 2022 at 13:00 SLT. These meetings are chaired by Vir Linden, and their dates and times can be obtained from the SL Public Calendar.

This is a summary of the key topics discussed in the meeting and is not intended to be a full transcript.

Official Viewers Update

No changes to the list of official viewer through until Thursday, June 16th, leaving them as:

• Release viewer: version 6.6.0.571939 – formerly the Performance Improvements viewer, dated May 25th – no change.
• Release channel cohorts (please see my notes on manually installing RC viewer versions if you wish to install any release candidate(s) yourself).
  • Nomayo Maintenance RC (Maintenance N) viewer, version 6.6.1.572179, June 1.
  • Makgeolli Maintenance RC viewer (Maintenance M) viewer, version 6.5.6.571575, May 12.
• Project viewers:
  • Performance Floater project viewer, version 6.5.4.571296, May 10.
  • Mesh Optimizer project viewer, version 6.5.2.566858, dated January 5, issued after January 10.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

General Viewer Status

• Bugs are still being ironed-out of the release version of the Performance Improvements viewer.
• It is hoped the Legacy Profiles project viewer will start to move forwards once more.
• There’s a further dedicated graphics project viewer in the wings, which may be appearing in the near future.

Materials and PBR Work

Please also see previous CCUG meeting summaries for further background on this work, with notes specific to the reflection probe work available in my week #22 and week #20 meeting summaries.

Outline of Work

• Work on an implementation of reflection probes which can be used both with PDR shading and with legacy content.
  • The overall aim of this work is to provide a means to support more physically accurate reflections in SL than can be currently generated (seen as a requirement for PBR support).
  • It applies to both PBR generated content, once available, and to legacy content.
• Creating a materials type with an associated inventory asset. This  will initially comprise the ability to copy a texture entry (with its specific parameters) to inventory, to be followed by initial work to work implement a PBR graphics pipe in the viewer.

Reflection Probes Progress

• Simulator-side support for reflection probes has been on test using an experimental view and simulator-side updates available on Aditi through the DRTSIM542 channel (Materials 1, Rumpus Room, Rumpus Room 2, and Materials Adult).

Materials Progress

• 

  Work is continuing on glTF materials import, and the hope is to have something that is functional within “the next week or two”. This uses the same unpacking mechanism as Unreal 4.

• Materials will be uploaded with an additional editor (shown right), which will allow some manipulation and delivery the materials as an inventory item when uploaded.
• Supported textures / capabilities:
  • RGB albedo + transparency.
  • RGB Occlusion/Roughness/Metalness: R = occlusion, G = roughness; Blue = metalness.
  • RGB emissive.
  • RGB normal (- alpha).
  • Double-sized supported (disables backface calling before issuing the draw call).
  • Two-sided lighting (so if the back of a triangle is visible, it flips the normal around).
• Functionality not initially supported will be the ability to change the UVU wrapping Mode (so everything will sill be repeat); no ability to change the metification / magnification filter per texture;
• The process separates the materials from the mesh, so the materials can’t know if things like tangents are present.
• Texture will initially have to be packed by a creator’s preferred toolset; once the project gets to a state of polishing, the importer should re-pack the textures itself, unless importing from a non-glTF source, in which case self-packing will still be required.
• Normals will likely be MikkTSpace, as per the glTF specification, but work needs to be done to see if supporting this could lead to clashes with the current normal maps rendering. This does mean that current Normal maps will not work on PBR materials.
• Uploads will be L$10 per texture, so L$40 if all four used.
• Brad Linden is working on getting the import into inventory working.

ALM Proposal / Work

At the #week #23 TPVD Developer Meeting (notes here), it was indicated that LL are “leaning” towards removal of the non-Advanced Lighting Model (ALM) (aka “forward rendering”) rendering path from the viewer, leaving just ALM rendering (aka deferred rendering”).

• This was triggered as a result of a bug report which initially appeared to suggest the Performance Improvements viewer unintentionally alters the render order of object faces. While it later proved that the issue was more an edge-case in the way a piece of content had been created, efforts to try to correct it and to ensure it rendered as desired in both ALM and non-ALM rendering, raised the question as to whether it would simply be easier to remove the non-ALM path.
• Were non-ALM rendering to be removed from the viewer, it would:
  • Only be done if it can be shown that this does not adversely impact performance (e.g. ALM runs roughly as well as non-ALM for those using the latter) on the broad cross-section of hardware most commonly operated by SL users.
  • Include a slider to manage the number of rendered local lights, as these are unlimited under ALM but can cause performance issues on low-end systems; thus a slider will help those on lower spec hardware to determine how many local lights they wish to have rendered.
  • Likely include a “data saving mode” that will prevent the download of materials for those on metered connections (to reduce the amount of data crossing their connection) and / or help those who find that materials loading can impact performance. This will have a UI warning that when employed, some objects may not look the way they are supposed to look.
• In terms of the last point above and “data saving mode” the focus is currently on how many users would need it *if* the Lab goes forward with the idea. This will help determine how much the mode is needed and how best to approach it.
• Given the ongoing work to support PBR and a more rounded set of materials, moving to deferred (ALM) rendering without fallbacks to non-ALM rendering – providing, again, the caveats noted above can be met / implemented – will help ensure a more reliable / consistent viewing experience.

Rigged Attachment Render Order

In terms of render order of object faces on rigged meshes, LL are considering adding a “render order number” that can be used by creators to ensure the orderly rendering of faces for transparent rigged attachments (thus allowing pre-loading of textures on “invisible” faces, etc.).

• This would hopefully overcome issues of implicit render ordering (which may change due to other viewer dependencies), and of interpretations of the rendering order by creators that can lead to the kind of issue noted in the ALM discussion above.
• There are complications in attempting this -e.g. render order is not per object, but per mesh, and is part of an overall schema for rigged attachments (attach order, linkset order, face index order); questions around overall outfit changes, where multiple attachments are being made, etc.
• Providing such an approach does not break masses of existing content, it would allow creators to continue to use the implicit ordering (and risk odd behaviours should LL make changes in the future that affect the order), or use the ordering system to explicitly set the order, no matter what changes might occur down the line.
• None of this work should impact the attachment order, but could significantly reduce the number of avatar draw calls.

Next Meeting

• Thursday June 30th, 2022.

The following notes were taken from my audio recording and chat log of the Content Creation User Group (CCUG) meeting held on Thursday, June 2nd 2022 at 13:00 SLT. These meetings are chaired by Vir Linden, and their dates and times can be obtained from the SL Public Calendar.

This is a summary of the key topics discussed in the meeting and is not intended to be a full transcript.

Official Viewers Update

• A new Maintenance RC viewer – Maintenance N, code-named Nomayo – version 6.6.1.572179 – was issued on June 1st. The viewer should offer improvements on media playback of web content, etc.

The rest of the official viewers remain as:

• Release viewer: version 6.6.0.571939 – formerly the Performance Improvements viewer, dated May 25th – NEW.
• Release channel cohorts:
  • Makgeolli Maintenance RC viewer (Maintenance M) viewer, version 6.5.6.571575, May 12.
• Project viewers:
  • Performance Floater project viewer, version 6.5.4.571296, May 10.
  • Mesh Optimizer project viewer, version 6.5.2.566858, dated January 5, issued after January 10.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

Reflection Probes

Additional information available within my week #20 CCUG meeting summary.

• A reflection probe will be a sphere or cube (mesh, prim or sculpt) set within a scene with specific properties allowing it to create a cube map of all objects within its bounding box. Anything within that bounding box with a “reflective” (shiny) surface (with the possible exception of worn attachments) will then offer “reflections” based on that cube map.
• Cube maps for probes are a purely viewer-side artefact, and are updated approx. once every 30 seconds at 60 fps, although this will “get smarter” in the future and be based on actual probe location (e.g. in or out of the current field of view, etc.).
• So, think of probes as mapping where reflections should be coming from in a scene, not as a tool for deciding which object should be reflecting things.
• The viewer will hopefully be able to handle up to 256 probes at any one time (the number of probes in a region can be unlimited), each with a (current) maximum effective draw distance of 64 m.
• Probes will:
  • Be detected by the viewer on load, much like lights already are (reflection probes use a lot of the code originally developed for light state management), with revisions to prevent issues associated with lighting such as flickering.
  • Have an influence volume, an ambience setting (which overrides the environment ambience) and a “near clip” parameter to help control what is rendered into the cube map (e.g. so items close to the centre of the probe and which might otherwise dominate any generated reflections, are not rendered into the cube map).
  • Require “PBR enabled”, and when this is set, legacy objects using glossiness and / or environment shine will render the reflection generated by a cube map respectively in accordance with the degree of glossiness / the sky environment, as well as any objects using the upcoming PBR materials.
    • There are a lot of nuances with the above bullet point such that legacy content won’t simply “work” with reflection probes, some degree of adjustment on object glossiness / shine might be required.
  • Work independently of LOD.
• Probes will not be designed for use as attachments.

In Brief

• Custom pivot point work: currently awaiting simulator updates & will require viewer-side changes; the focus is slowly moving towards trying to move the latter part of the work forward “in the not too distant future”.
• It was noted that the mesh optimiser (as in the current release viewer) still has issues that need to be addressed,
  • One such issue is when the LOD generator is implicitly asked to simplify a model, and it cannot hit a target without destroying UV seams, it will destroy the UV seams (whereas GLOD will delete triangles, leaving holes in models). This appears to be happening on models with a lot of UV islands, resulting in texels becoming visible when they should never be drawn.
  • LL acknowledges that neither the optimiser issue nor the GLOD issue are ideal.
  • Issues with the mesh uploader are requested via Jira with objects included, if possible.
• In response to a question, LL currently have no plans to alter the LI calculations for Animesh, however, with the performance improvements (and given part of the LI “penalty” was to compensate for the performance hit of animating Animesh characters), there is a suggestion it should perhaps be re-visited, particularly given most at the meeting with an opinion felt the 15LI penalty was a “blocker” to developing Animesh content.
• The above points led to a general discussion on LODs, decimation, Land Impact based on size, the ARCTan project (still something LL would “like to get back to”), the impacts of draw calls over poly counts, etc., but with no actionable points raised.
• There is a fork in the texture rendering pipeline underdevelopment that should, once available, ensure that only the required texture resolution is loaded when it is required (e.g. so the tiny buttons and the little broach and the earring etc., on an avatar won’t all be displayed at 1024×1024 all the time, but the system will only ever download at use a 128×128 version of the textures used).
• Work is also in progress to ensure the official viewer uses all available video memory. This will eventually see the VRAM slider in Preferences → Graphics vanish.
  • Note that this is available video memory – not necessarily the total on your system; obviously, running SL and multiple browser tabs and other windows will impact how much memory the viewer can actually access, leading to the potential of textures being uploaded.

Next Meeting

• Thursday June 16th, 2022.

Lemon Trees Mediterranean – blog post

The following notes were taken from my audio recording and chat log of the Content Creation User Group (CCUG) meeting held on Thursday, May 19th 2022 at 13:00 SLT. These meetings are chaired by Vir Linden, and their dates and times can be obtained from the SL Public Calendar.

This is a summary of the key topics discussed in the meeting and is not intended to be a full transcript.

Official Viewers Update

• On Wednesday, May 18th, the Performance Improvements RC viewer updated to version 6.6.0.571869.

The rest of the official viewers remain as:

• Release viewer: version version 6.5.5.571282, – formerly the MFA RC viewer, dated April 26, promoted Wednesday, May 4th – Non change.
• Release channel cohorts:.
  • Makgeolli Maintenance RC viewer (Maintenance M) viewer, version 6.5.6.571575, May 12.
• Project viewers:
  • Performance Floater project viewer, version 6.5.4.571296, May 10.
  • Mesh Optimizer project viewer, version 6.5.2.566858, dated January 5, issued after January 10.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

Materials and PBR Work

Please also see previous CCUG meeting summaries for further background on this work. In summary:

Three core elements of work:

• Work on an implementation of reflection probes which can be used both with PDR shading and with legacy content. This formed the focus of this meeting.
  • The overall aim of this work is to provide a means to support more physically accurate reflections in SL than can be currently generated (seen as a requirement for PBR support).
  • It applies to both PBR generated content, once available, and to legacy content.
• Foundational work in creating a materials type with an associated inventory asset, as per the week #16 meeting. This  will initially comprise the ability to copy a texture entry (with its specific parameters) to inventory, to be followed by
• Initial work to work implement a PBR graphics pipe in the viewer.

Reflection Probes

• Additional information available within the week #18 CCUG meeting summary.
• This work is close to feature complete.
  • The viewer gets to work with 256 reflection probes, which take the form of spheres or boxes within a region.
  • Anything within a sphere or box will receive reflections from the cube map rendered from the centre of the sphere / box.
  • Some of these probes will be automatically placed in open areas of land where there are objects, etc., by the viewer.
  • Additional probes can be created by users using prims tagged as probes and placed where they want to influence the reflections being generated (e.g. inside rooms, etc.).
  • Baking for reflection probes will be automatic, and updates will be handled at least once every 30 seconds.
• There is a performance hit with the capability, and this is still being adjusted so that it will hopefully not be overly onerous.
• Elizabeth Jarvinen (Polysail) is working  on the current light shader to enable legacy content to receive the reflection probes without looking “too different” and look like it belongs in the environment along with PBR content.

Materials /PBR Work

• Progress continues in developing a “materials” type with an associated inventory asset capable for containing PBR materials data.
  • LSL access to said materials is regarded as being “tricky”, simply because the materials will be an asset type loaded by the viewer.
  • What is being proposed is to have the ability to “override” elements of the asset (e.g. colour or texture) via LSL by applying the changes to the properties of the object face to which the materials is applied.
    • So, for example, the LSL override says, “OK. I know this material has a texture UUID inside it – I don’t know what it is, but I want this face to use MY texture UUID instead” – so the material asset itself is not changed / updated, but the UUID defined by the LSL code is displayed, rather than the texture UUID defined by the asset.
    • If the materials asset type subsequently be changed, then the overrides applied via LSL to the object face are automatically dropped until such time as new overrides are applied.
  • This is seen as the most flexible approach, as it protects the integrity of the materials asset (in a similar manner to texture data) whilst also allowing the flexibility of using colour variants against an asset type (such as in the case of a sweater using a single materials asset, but with multiple colour options in the pack or in allow a HUD to alter the tint of an object that uses a materials asset).
• Nothing of significance to report on the PDR shader work.

In Brief

• Custom pivot point work: currently awaiting simulator updates & will require viewer-side changes.
• A fix has been implemented in the viewer to speed-up opening media / web floaters (such as search). This should be surfacing in the next Maintenance RC viewer (“Maint N” to follow the Makgeolli  Maintenance RC).
  • An upcoming simulator release should have a fix for objects failure to rez when users first log-in. .

Next Meeting

• Thursday June 2nd, 2022.

The following notes were taken from my audio recording and chat log of the Content Creation User Group (CCUG) meeting held on Thursday, May 4th 2022 at 13:00 SLT. These meetings are chaired by Vir Linden, and their dates and times can be obtained from the SL Public Calendar.

This is a summary of the key topics discussed in the meeting and is not intended to be a full transcript.

Official Viewers Update

• On Wednesday, May 4th, the de facto release viewer was updated to version 6.5.5.571282, formerly the MFA RC viewer, dated April 26. See my notes on this viewer here.
• On Thursday, May 5th, the Performance Improvements RC viewer updated to version 6.6.0.571507.
  • The focus for this viewer is now bug and crash fixes, rather than adding functionality.
  • This update also sees the viewer merged with the release viewer MFA code base, and so will require anyone who uses MFA and who has not logged-in to Second Life using an MFA token to do so -follow the link above for details.

The rest of the official viewers remain as:

• Release channel cohorts:
  • Makgeolli Maintenance RC viewer (Maintenance M) viewer, version 6.5.5.570983, April 26.
• Project viewers:
  • Performance Floater project viewer, version 6.5.4.569531, March 18.
  • Mesh Optimizer project viewer, version 6.5.2.566858, dated January 5, issued after January 10.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

Graphics Update

• During the previous CCUG meeting, the potential direction for Physically Based Rendering (PBR) materials support was discussed at length, and concerns were raised that for PBR to be effective, SL’s lighting / reflections model would need to be updated.
• As a result, these branches of work have commenced:
  • The foundational work in creating a materials type with an associated inventory asset, as per the week #16 meeting. This  will initially comprise the ability to copy a texture entry (with its specific parameters) to inventory.
  • Foundational work on the PBR graphics pipe in the viewer.
  • Work on an implementation of reflection probes which can be used both with PDR shading and with legacy content. This formed the focus of this meeting.

Reflection Probes

• Runitai Linden envisages two forms of reflection probes:
  • Those that are part of a scene’s environment and are automatically placed, but which can (hopefully) be adjustable by scene builders – this is the current focus of the work, with the reflection probes married to the octree so they will align with octree nodes.
  • Reflection probes which can be specifically placed within structures by their creators (e.g. a probe within each room of a house) or which can be created and added to a legacy structure.
    • These would override “environment reflection probes” to prevent clashes (so that, for example, your nice shiny kitchen worktop is not reflecting the sky outside, but is offering reflections of the static objects on and around it.
    • Their size would be based on the volume they are attached to, which would allow for parallax correction, allowing the reflections to be accurately generated within a room space.

• These probes:
  • Comprise an object (prim) defined as a reflection probe which essentially uses the 360º snapshot code viewer to produce a cube map of its surroundings with each face set to 512×512 resolution (so 6 x 512×512), which can then be used to generate object “reflections” on surfaces within their volume of control.
    • This will require some additional checkboxes to be added to the build floater / viewer, but otherwise means probes can be inserted into “indoor” scenes using the existing build toolset.
  • Capture everything except avatars (so no reflections of your avatar walking past a shiny surface).
  • Initially trigger the generation of their cube maps by the viewer on entering a scene.
    • Maps are cached on disk, with up to 64 held in memory.
    • Currently only 8 maps are active at any one time. However, given sampling of just 8 maps for rendering can take up to 70% of an RTX 3080 GPU, this number is likely to be dropped to 4, unless a more targeted means of sampling for a given pixel can be determined.
  • Are updated whenever the number of objects in them changes or the Day Cycle changes, although the update rate is no more than one probe per frame.
  • Include Fresnel calculations when generating cube maps.

• The overall aim of this work is to provide a means to support more physically accurate reflections in SL than can be currently generated (seen as a requirement for PBR support). This work can then both:
  • Be integrated into the PBR work as that is added to Second Life.
  • Improve the existing SL environment map so that legacy content does not clash with PBR content as badly as might otherwise be the case – although it will not be a perfect blending of the two.
  • However, it will be “very, very hard for this to not break [some] content”, although the overall result should improve so SL looks.
  • Given this, the capability will need a lost of assistance with testing.
• Concern has been raised about how to limit / control the use of probes so as not to cause potential performance issues (e.g. by creators including a probe with all of their products that have a shiny / reflective surface).
  • The plan is to have every single pixel in a rendered scene covered by at least one probe via the automatic placement, so objects that are outside certainly should not need their own reflection probes.
  • Even so, it has been suggested that time is taken to produce a “best practices” guide to using reflection probes and to provide examples of bad probe set-ups.
• It was also suggested that a flag for “do not reflect” should be provided to help control what is included in a cube map in a given setting. It was not clear if this would be possible, so the question was set aside for the next meeting.
• General Notes:
  • This work is still in its early stages, and elements may change.
  • Several aspects of reflection generation have yet to be tackled (e.g. handling 100% transparent and 100% alpha textured surfaces).
  • The work has yet to be tied-into the existing graphics controls in the viewer so that everything works consistently and logically.
  • Some of the additional work required for generating reflections might be compensated for by optimising other elements of the shader system as this work progresses.
  • At some point this work will be available within a project viewer for testing on Aditi.

Note: while reflection probes formed the core of this meeting, this does not mean this work will be shipped ahead of the more PBR-related work. All three project branches are in their early stages, and so no decision on prioritisation among them has been taken.

PBR-Specific Work

• The PBR graphics pipe is still being plumbed-in.
• No work has really started with the PBR shader set.
• The plan is still to follow the glTF specification.
  • As this embraces the PBR Metallic Roughness workflow, this will be the first to be supported.
  • If demand surfaces, LL might add Specular Glossiness as a follow-up.

Next Meeting

• Thursday May 19th, 2022.