Category: SL User Group Meetings

2020 SUG meeting week #45: further uplift update

Posted on by Inara Pey
A Thousand Windows, September 2020 – blog post

The following notes were taken from the November 3rd Simulator User Group meeting.

Server Updates and Cloud Uplift

Please reference to the server deployment thread for the latest updates.

  • There are no planned deployments to the simulators running on the Lab’s core SLS channel.
  • RC deployments are planned as follows:
    • On Wednesday, 4th November all simulators on the LeTigre and BlueSteel RC channels should become AWS hosted.
    • On Thursday, 5th November all simulators on the Magnum RC channel should also become AWS hosted.
    • However, at the time of the meeting, it was not clear if all of the RC channels would be running the same version of simulator software.

The current work on migration is such that as per Private Regions Available in Limited Quantity (via Linden Lab), private regions are once more being made available.

SL Viewer

The Start of the week saw the following viewer updates on Monday, November 2nd:

  • The Jellydoll project viewer updated to version 6.4.11.551213.
  • Custom Key Mappings project viewer updated to version 6.4.10.549685.

The rest of the official viewers in the pipelines remain as follows:

  • Current release viewer version 6.4.10.549686, formerly the Mesh Uploader RC promoted on October 14 – No Change.
  • Release channel cohorts:
    • Cachaça Maintenance RC viewer, version 6.4.11.551139, issued October 27.
  • Project viewers:
    • Legacy Profiles viewer, version 6.4.11.550519, October 26.
    • Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
    • Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
    • 360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.

In Brief

  • Group Chat: LL deployed updates to the group chat service in an attempt to relieve at least some of the issues that groups have been experiencing over the last several weeks. Testing has suggested the group chat sessions should be faster and more reliable than has been experienced within some groups (notably those with large memberships). However, the issue remains open pending further observation / feedback.
  • TP failures continue, and are being noted by the Lab, although not at the rates at which users appear to be experiencing them.
    • However, the nature of the beast means that at present, correlation of all the logs involved in a teleport has to be done manually, and this is impacting the Lab’s ability to arrive at a potential root cause (or causes).
    • Once the majority of cloud migration work has been completed, and if the matters hasn’t been resolved, Simon Linden may set-up another round of TP testing by users as we’ve seen in past issues of teleport issues.

2020 TPVD meeting week #44: summary with uplift news

Posted on by Inara Pey
The Muse, September 2020 – blog post

The following notes are taken from the TPV Developer meeting held on Friday, October 30th, 2020. These meetings are generally held every other week, unless otherwise noted in any given summary. The embedded video is provided to Pantera – my thanks to her for recording and providing it. Time stamps are included with the notes will open the video at the point(s) where a specific topic is discussed. Note these summaries are not intended to be a full reporting on all topics discussed, but focus on those items that are more directly user-facing.

Another exceptionally brief meeting, with some discussion in chat, so please refer to the video as well.

Cloud Uplift

[0:00-1:01]

  • The transitioning of regions to running on AWS services in progressing “extremely well”.
  • So far, LL has been able to deal with those issues that have arisen.
  • At the time of writing, just under 15% of the the main (Agni) regions are now running on AWS.
  • This amount is set to increase “significantly” in week #45 (commencing Monday, November 2nd).
  • If there are issues users are encountering with regions running on AWS that haven’t been reported, now is the time to test them (preferably with the official viewer) to confirm they can be reproduced and then report them via the Second Life Jira.

SL Viewer News

[1:06-2:48]

  • Current release viewer version 6.4.10.549686, formerly the Mesh Uploader RC promoted on October 14 – No Change.
  • Release channel cohorts:
    • Cachaça Maintenance RC viewer, version 6.4.11.551139, issued October 27.
  • Project viewers:
    • Project Jelly project viewer (Jellydoll updates), version 6.4.10.549690, October 1.
    • Custom Key Mappings project viewer, version 6.4.5.544079, June 30.
    • Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
    • Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
    • 360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.
    • legacy Profiles project viewer, version 6.3.2.530836 – but see below.

General Viewer Notes

  • The Legacy Profiles project viewer appears to be in a state of flux:
    • Updates to the viewer were dependent on on back-end changes which were in turn dependent on completion of the cloud migration work.
    • It had been hoped that an interim workaround could be made to allow the viewer to progress without the back-end changes.
    • On Monday, October 26th, the viewer was apparently updated to version 6.4.11.550519 (including on the Alternative viewers web page), presumably to bring it to par with the release viewer code base.
    • By Friday, October 30th, this appears to have been rolled-back to version 6.3.2.530836 (noted above), dated September 2019.
    • [10:07-10:35] When released, this viewer will see the deprecation of all aspects of the current web profiles, other than the Feed.
    • [11:02-11:36] The Feed will be made available through the viewer “as is”, but may be enhanced or possibly retired in the future.
  • An upcoming project / RC viewer will feature a replacement for the VFS (Virtual File System) cache.
    • There may be some benefits from the initial release of this viewer, however its primary aim is to get a new cache framework in place for upcoming cache-related projects (e.g. texture fetching and caching).

In Brief

  • [2:54-3:37] Firestorm currently have a pre-release of their EEP Beta viewer in testing, and are moving to promote that viewer to full release status, and is moving towards a code freeze so that it can progress to a release.
  • [4:19-5:13] BUG-229555 “[CEF 2020] AltGr key doesn’t work within internal web browser” – this issues also apparently affects special characters on windows as well. It has been accepted by LL as an internal Jira, and a fix should be in an upcoming Maintenance viewer / update.

2020 CCUG meeting week #44 summary

Posted on by Inara Pey
Poughkeepsie, September 2020 – blog post

The following notes were taken from my audio recording and chat log of the Content Creation User Group (CCUG) meeting held on Thursday, October 29th 2020 at 13:00 SLT. These meetings are chaired by Vir Linden, and agenda notes, meeting SLurl, etc, are are available on the Content Creation User Group wiki page.

There was a lot of general discussion about animation synchronisation, cloud uplift (the majority of which I’ve already reported in my SUG and TPVD meetings of late), etc., none of which should out as really reportable.

Jellydoll / ARCTan

  • Vir has been working on some bug fixes to the Project Jelly viewer (version 6.4.10.549690 at the time of writing), and this work is now with the Lab’s QA team. When issues, this will see the viewer also based on the current release viewer code base.
  • As a reminder:
    • The current ARCTan work is focused on the viewer-side updates to avatar complexity calculations.
    • Work on providing in-world object rendering costs (LOD models, etc.) which might affect Land Impact will be handled as a later tranche of project work, after the avatar work.

Graphics

  • The Graphics team is still looking at options for replacing OpenGL, particularly as a result of Apple’s plans to deprecate all support for it on their systems.
  • The OpenGL project is large-scale, so the Graphic team is also looking at other shorter-term projects related to performance improvements. These may include:
    • Possible optimisations for people on low-end systems (as noted in my previous CCUG summary, roughly 1/3 of all users are running on systems that do not support Vulkan and many of these systems are more than 5 years old).
    • Linden Water: since the last TPV meeting (see: 2020 TPVD meeting week #42: summary) the Lab believe they have determined that the FPS loss seen when rendering Linden Water appears to be related to a combination of the actual rendering and with water reflections. This is still being investigated. One idea to deal with the latter is to completely disable water reflections in the viewer; however, doing this at present affects the sky and the look of the Sun, so again,further investigation is needed.
  • In addition, the graphics team is looking to see if the viewer UI rendering can be divorced from other aspects of viewer rendering to see if any performance improvements can be obtained by keeping the two separate – currently, a lot of cycles are taken up in the drawing of things like UI panels and buttons.

In  Brief / General

  • Teleport failures are still under investigation. Beq Janus from Firestorm is also investigation, and has encountered a situation where the failure may be related to the manner in which regions are queried bases on draw distance, and the system potentially becomes confused as to the actual destination.
  • Megaprims and relaxing linking constraints (BUG-229551):
    • As  expressed by the feature request, the Lab sees this as being two issues: allowing the use of prims larger than 64m on a size / diameter, and handling region surrounds.
    • In particular, the Lab is looking at the potential of braking out region surrounds (“sim surrounds”) as a supported feature in their own right, simply because of the size of the use case. One suggestion for this is to have what is seen beyond a the edges of an individual region to be definable as texturing, rather than just the default of Linden Water.
    • Increasing prim sizing beyond the current maximum is seen as problematic as the size constraint touches on multiple aspects of the simulator code (e.g. Interest Lists being one) that would have to be overhauled – which is not something the Lab wants to undertake in the near future, although this does do preclude the potential for very large prim sizes at some point.
  • The cloud migration work has left the Lab with a certain amount of “technical deficit” – work  deferred in favour of simply getting systems and services transitioned to running on AWS infrastructure and hardware. This means that once the Uplift Project is completed, there will be a number of areas of simulator and back-end services that will need to be re-visited.
  • Potential projects to follow-on projects outside of the graphics performance updates mentioned above could include:
    • Updating the default terrain texture.
    • Further work on easing the new user experience,
    • Further work on viewer UI improvements.
    • Streaming changing an avatar’s appearance.
  • Bone uploads: the question was asked that why, given Bento provides 120+ bones, single avatar sub-mesh uploads are limited to a maximum of 110 bones. The reason is that 110 bones is the maximum some graphics systems can handle in a single upload before things go wrong.

Date of Next Meeting

2020 SUG meeting week #44: further uplift update

Posted on by Inara Pey
Boulder, September 2020 – blog post

The following notes were taken from the October 20th Simulator User Group meeting.

Server Updates and Cloud Uplift

  • There are no planned deployments to the regions running within the Lab’s co-lo facilities.
  • Starting on Tuesday, October 27th, all regions on the “AWS channel” will be updated with a simulator release  – version 551155 – that includes extended logging capabilities and a range of fixes, as indicated in the release notes.

For details of the deployment of main grid regions running via AWS cloud services, please see the following:

Issues

The grid has seen a range of issues coincident to the uplift work that may or may not be related to that work (as they are occurring across regions that are both now running on AWS and those still running at the Lab’s co-lo facility, although some appear to happen more frequently within AWS region.) The latest of these is object failing to rez when pulled from inventory. However, Mazidox Linden indicated that LL believe they have an handle on this issue, and the AWS region simulator version may have address it.

The most significant  issues (again which may or may not be related to the cloud uplift) being experienced comprise:

  • Group chat failures.
  • Teleport failures.
  • Rezzing issues.
  • Slow opening of legacy profiles.
  • Slow opening on the World Map (hopefully fixed with the 551155 release).

A further issue that has been noted, but yet to be confirmed, is that idle scripts appear to be taking up to to between 16% and 26% more script timing on regions running on AWS when compared to regions still operating out of the Labs co-lo facility.

Uplift Status

  • As per April Linden’s blog post, around 1,000 region on the main grid that are running on AWS.
    • Thus far, LL have only seen minor issues from their perspective, with those regions are running with “only minor hiccups”.
  • The deployment this week will see the total number of regions on AWS increase to between 2,000-2,500.
  • The updates required to allow the simulator to run on AWS systems represents – in Mazidox Linden’s words – “”the largest change to the simulator [software]  ever.”
  • Maxidox also confirmed there is an upcoming change that will see simulators running on AWS given a synthetic hostname rather than the viewer just giving the IP address of the AWS compute node in URI form.  The synthetic hostname will map to how LL describes simhosts internally.

SL Viewer

The Start of the week has seen the following viewer updates:

  • The Cachaça Maintenance RC viewer updated to version 6.4.11.551139 on Tuesday, October 27th, 2020.
  • The Legacy Profiles Viewer updated to version 6.4.11.550519 on Monday, October 26th, 2020.

The rest of the official viewers in the pipelines remain as follows:

  • Current release viewer version 6.4.10.549686, formerly the Mesh Uploader RC released October t and promoted on October 14 – No Change.
  • Project viewers:
    • Project Jelly project viewer (Jellydoll updates), version 6.4.10.549690, October 1.
    • Custom Key Mappings project viewer, version 6.4.5.544079, June 30.
    • Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
    • Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
    • 360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.

In Brief

  • Some users (notably Firestorm users) are claiming that uplift is “breaking” region crossings, and that Firestorm should offer a higher default bandwidth allowance. However:
    •  The bandwidth setting applies to UDP only – the majority of data related to avatars, attachments and objects are handled via HTTP through the CDN, over which the UDP bandwidth has no influence.
    • While UDP messaging is involved in region crossings, changing the bandwidth default is viewed as potentially unwise whilst the uplift for is in progress, as it would largely be impossible to objectively tell if the change has improved matters.

2020 SUG meeting week #43: further uplift update

Posted on by Inara Pey
Bungenäs, Binemust – blog post

The following notes were taken from the October 20th Simulator User Group meeting.

Cloud Uplift Update

There are no formal simulator deployments for week #43. This is to make way for selected regions (both RC and Main channel) to be migrated to AWS services (the cloud). This means those regions that are transitioned will be restarted.

For details of the initial announce on the uplift, please refer to Lab expanding number of regions on AWS.

Speaking at the simulator User Group meeting on October 20th, Maxidox Linden provided the following update:

This morning we started our medium-size (at least by the size of a world) movement of regions to cloud based simhosts. Specifically we now have over 300 simulators running in the cloud. We’re looking at how quickly we can move more, as well.
For this round we went with a mixture of extremely high impact regions and extremely low impact “filler” regions, to stress various parts of our systems like the region allocation process and how we pack regions into a host.
In future rounds it’s likely we’ll be focusing on specific Release Candidate channels.
Basically: My *goal* (I’m not saying it’s what we’re doing, but I’m saying it’s what I’m trying to do) is to make this the worst it gets, and it only gets better from here.

– Mazidox Linden

Bullet Notes on Uplift

  • As of the October 20th SUG meeting, some 300 regions have been transitioned to the cloud, representing a mix of region in terms of use.
  • Performance issues have been warned about because there are a number of back-end systems that have yet to be uplifted (and presumably need the simulators in place before they can be), which may have some impact.
  • The hope is that some of these additional services will be transitioned when they can, rather than being “left until last”.
    • One of those earmarked for transition that has been causing some angst are the servers managing the KVP database for experiences.  There have already bee reports of degraded experience performance on AWS-hosted regions, so the hope is to get the KVP database moved, with “quiet a bit of effort” going into it.
  • Those who have had regions uplifted and feel they are suffering adversely from the move can request a roll back to being hosted in the Lab’s co-lo. However:
    • As all regions will be running on AWS services by year’s end, LL would rather region holders bear with any issues they may encounter whilst things are in a state of flux, and if they are encountering specific issues, to work with the Lab to try and identity and rectify matters.
    • Those who feel they need a roll-back to the lab’s co-lo facility should file a support ticket, providing a clear indication of why they would like their region(s) rolled back, and the issues they are encountering.
During the transition (as now) either the uplifted or data centre regions will have some penalty … we expect that will mostly disappear by the time we’re done

– Oz Linden on possible simulator (and other) performance impacts
during region migration

  • The Lab’s aim is to continue to make the transitioning of services from their co-lo to AWS services as invisible to users as possible (that is, you shouldn’t really be able to tell the difference between a service now running on AWS and when it was running via the Lab’s co-lo).

SL Viewer

There have been no updates to the current list of official viewers, leaving the pipelines as follows:

  • Current release viewer version 6.4.10.549686, formerly the Mesh Uploader RC released October t and promoted on October 14 – No Change.
  • Release channel cohorts:
    • Cachaça Maintenance RC viewer, version 6.4.10.549752, issued October 1.
  • Project viewers:
    • Project Jelly project viewer (Jellydoll updates), version 6.4.10.549690, October 1.
    • Custom Key Mappings project viewer, version 6.4.5.544079, June 30.
    • Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
    • Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
    • Legacy Profiles viewer, version 6.3.2.530836, September 17, 2019. Covers the re-integration of Viewer Profiles.
    • 360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.

In Brief

  • Group chat issues are being worked on, but is pending the uplift work.
  • The long-promised increase in sound loop support from 10 seconds to 30seconds, first promised a fair while ago now, is … pending the uplift work – although it is rumoured (and subject to confirmation) to be a Premium Plus benefit.

2020 TPVD meeting week #42: summary

Posted on by Inara Pey
Isolation’s Passengers – blog post

The following notes are taken from the TPV Developer meeting held on Friday, October 16th, 2020. These meetings are generally held every other week, unless otherwise noted in any given summary. The embedded video is provided to Pantera – my thanks to her for recording and providing it. Time stamps are included with the notes will open the video at the point(s) where a specific topic is discussed. Note these summaries are not intended to be a full reporting on all topics discussed, but focus on those items that are more directly user-facing.

Another exceptionally brief meeting, with some discussion in chat, so please refer to the video as well.

SL Viewer News

[0:15-2:18]

As per my CCUG meeting notes, the Mesh Uploader RC was promoted this week to de facto release status – refer to the release notes (link below) for details of the cosmetic and informational changes made to the uploader (which are in part based on contributions by Beq Janus of the Firestorm team). This leaves the current official viewer release list as follows:

  • Current release viewer version 6.4.10.549686, formerly the Mesh Uploader RC released October t and promoted on October 14 – NEW.
  • Release channel cohorts:
    • Cachaça Maintenance RC viewer, version 6.4.10.549752, issued October 1.
  • Project viewers:
    • Project Jelly project viewer (Jellydoll updates), version 6.4.10.549690, October 1.
    • Custom Key Mappings project viewer, version 6.4.5.544079, June 30.
    • Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
    • Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
    • Legacy Profiles viewer, version 6.3.2.530836, September 17, 2019. Covers the re-integration of Viewer Profiles.
    • 360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.

General Viewer Notes

  • As per this week’s CCUG meeting, Vir is hopeful that the Legacy Profiles project viewer may be able to move forward soon using a workaround to get it past the current blocker of needing some web-side updates.
  • The Custom Key Mappings project viewer is being considered from promotion to RC status with its next update.
  • There is an upcoming RC viewer featuring some UI changes – possibly including the update to give HUD items in inventory their own inventory icon, as discussed by Steeltoe Linden in the CCUG week #40 meeting. Thus might be appearing in the next week or so.
The proposal UI change to distinguish HUD items from other objects in inventory.Left: how things are now, with HUDs using the same cube icon as other objects. Right: the proposed new HUD cog icon.
  • [5:24-6:54] There have been some issues with viewer documentation not appearing on the relevant web pages, leading to concerns that such documentation might be “going away”.
    • The problem has actually been caused by the mechanism used to generate the information not having been updated for some time (and also not even using current information), so when it was migrated to AWS services, it “fell apart”.
    • Rather than try to fix the issue, the decision has been made so that once uplift has been completed, the mechanism will be replaced with an easier to maintain process. However, it does mean that for the time being, some viewer related documentation may not be available.

In Brief

  • [9:20-10:00] BUG-228679: Media-on-a-Prim doesn’t register keystrokes correctly on Mac – unfortunately, no progress to report.
  • [12:12-13:40] A couple of TPVs have been holding off making a formal release of the Environment Enhancement Project (EEP) updates beyond their current beta releases in case LL make further EEP update releases like the recent Love Me Render #4 viewer release. However, the advice from the Lab to these TPVs is:
    • To consider the high-priority EEP fixes as being “done”, as the Graphics team is now focused on priority long-term architecture work (including moving the viewer beyond OpenGL).
    • That future EEP fixes, when made, will form a part of the regular Maintenance RC viewer releases, rather than having any dedicated RC viewer updates.
  • [13:49-16:58] EEP performance issues:
    • Some TPVs have reported significant performance / FPS drop-off with EEP (around 15%). However, the Lab is not seeing anything close to this with the official viewer.
    • Unless there are detailed reports filed on the issue – using the official release viewer – that provide information on where / when / how the issue is being witnessed (e.g. the kind of test carried out, where in-world, the hardware configuration and viewer settings used, the results obtained, etc.), that allow LL to reproduce the problems, they do not see what can be done by way of investigation / fixes.
      • It was pointed out that comparable testing on the official viewer is difficult due to the limited amount of video memory available (512 MB compared to a minimum of 1 GB on most TPVs), which also affects performance, and so might skew results / how results are seen (e.g. are they EEP or are they due to texture load issues?).
      • Vir indicated that the amount of assigned texture memory is something the Lab will be looking at in the future.