2020 SUG meeting week #44: further uplift update

Boulder, September 2020 – blog post

The following notes were taken from the October 20th Simulator User Group meeting.

Server Updates and Cloud Uplift

  • There are no planned deployments to the regions running within the Lab’s co-lo facilities.
  • Starting on Tuesday, October 27th, all regions on the “AWS channel” will be updated with a simulator release  – version 551155 – that includes extended logging capabilities and a range of fixes, as indicated in the release notes.

For details of the deployment of main grid regions running via AWS cloud services, please see the following:

Issues

The grid has seen a range of issues coincident to the uplift work that may or may not be related to that work (as they are occurring across regions that are both now running on AWS and those still running at the Lab’s co-lo facility, although some appear to happen more frequently within AWS region.) The latest of these is object failing to rez when pulled from inventory. However, Mazidox Linden indicated that LL believe they have an handle on this issue, and the AWS region simulator version may have address it.

The most significant  issues (again which may or may not be related to the cloud uplift) being experienced comprise:

  • Group chat failures.
  • Teleport failures.
  • Rezzing issues.
  • Slow opening of legacy profiles.
  • Slow opening on the World Map (hopefully fixed with the 551155 release).

A further issue that has been noted, but yet to be confirmed, is that idle scripts appear to be taking up to to between 16% and 26% more script timing on regions running on AWS when compared to regions still operating out of the Labs co-lo facility.

Uplift Status

  • As per April Linden’s blog post, around 1,000 region on the main grid that are running on AWS.
    • Thus far, LL have only seen minor issues from their perspective, with those regions are running with “only minor hiccups”.
  • The deployment this week will see the total number of regions on AWS increase to between 2,000-2,500.
  • The updates required to allow the simulator to run on AWS systems represents – in Mazidox Linden’s words – “”the largest change to the simulator [software]  ever.”
  • Maxidox also confirmed there is an upcoming change that will see simulators running on AWS given a synthetic hostname rather than the viewer just giving the IP address of the AWS compute node in URI form.  The synthetic hostname will map to how LL describes simhosts internally.

SL Viewer

The Start of the week has seen the following viewer updates:

  • The Cachaça Maintenance RC viewer updated to version 6.4.11.551139 on Tuesday, October 27th, 2020.
  • The Legacy Profiles Viewer updated to version 6.4.11.550519 on Monday, October 26th, 2020.

The rest of the official viewers in the pipelines remain as follows:

  • Current release viewer version 6.4.10.549686, formerly the Mesh Uploader RC released October t and promoted on October 14 – No Change.
  • Project viewers:
    • Project Jelly project viewer (Jellydoll updates), version 6.4.10.549690, October 1.
    • Custom Key Mappings project viewer, version 6.4.5.544079, June 30.
    • Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
    • Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
    • 360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.

In Brief

  • Some users (notably Firestorm users) are claiming that uplift is “breaking” region crossings, and that Firestorm should offer a higher default bandwidth allowance. However:
    •  The bandwidth setting applies to UDP only – the majority of data related to avatars, attachments and objects are handled via HTTP through the CDN, over which the UDP bandwidth has no influence.
    • While UDP messaging is involved in region crossings, changing the bandwidth default is viewed as potentially unwise whilst the uplift for is in progress, as it would largely be impossible to objectively tell if the change has improved matters.

2020 SUG meeting week #43: further uplift update

Bungenäs, Binemust – blog post

The following notes were taken from the October 20th Simulator User Group meeting.

Cloud Uplift Update

There are no formal simulator deployments for week #43. This is to make way for selected regions (both RC and Main channel) to be migrated to AWS services (the cloud). This means those regions that are transitioned will be restarted.

For details of the initial announce on the uplift, please refer to Lab expanding number of regions on AWS.

Speaking at the simulator User Group meeting on October 20th, Maxidox Linden provided the following update:

This morning we started our medium-size (at least by the size of a world) movement of regions to cloud based simhosts. Specifically we now have over 300 simulators running in the cloud. We’re looking at how quickly we can move more, as well.
For this round we went with a mixture of extremely high impact regions and extremely low impact “filler” regions, to stress various parts of our systems like the region allocation process and how we pack regions into a host.
In future rounds it’s likely we’ll be focusing on specific Release Candidate channels.
Basically: My *goal* (I’m not saying it’s what we’re doing, but I’m saying it’s what I’m trying to do) is to make this the worst it gets, and it only gets better from here.

– Mazidox Linden

Bullet Notes on Uplift

  • As of the October 20th SUG meeting, some 300 regions have been transitioned to the cloud, representing a mix of region in terms of use.
  • Performance issues have been warned about because there are a number of back-end systems that have yet to be uplifted (and presumably need the simulators in place before they can be), which may have some impact.
  • The hope is that some of these additional services will be transitioned when they can, rather than being “left until last”.
    • One of those earmarked for transition that has been causing some angst are the servers managing the KVP database for experiences.  There have already bee reports of degraded experience performance on AWS-hosted regions, so the hope is to get the KVP database moved, with “quiet a bit of effort” going into it.
  • Those who have had regions uplifted and feel they are suffering adversely from the move can request a roll back to being hosted in the Lab’s co-lo. However:
    • As all regions will be running on AWS services by year’s end, LL would rather region holders bear with any issues they may encounter whilst things are in a state of flux, and if they are encountering specific issues, to work with the Lab to try and identity and rectify matters.
    • Those who feel they need a roll-back to the lab’s co-lo facility should file a support ticket, providing a clear indication of why they would like their region(s) rolled back, and the issues they are encountering.
During the transition (as now) either the uplifted or data centre regions will have some penalty … we expect that will mostly disappear by the time we’re done

– Oz Linden on possible simulator (and other) performance impacts
during region migration

  • The Lab’s aim is to continue to make the transitioning of services from their co-lo to AWS services as invisible to users as possible (that is, you shouldn’t really be able to tell the difference between a service now running on AWS and when it was running via the Lab’s co-lo).

SL Viewer

There have been no updates to the current list of official viewers, leaving the pipelines as follows:

  • Current release viewer version 6.4.10.549686, formerly the Mesh Uploader RC released October t and promoted on October 14 – No Change.
  • Release channel cohorts:
    • Cachaça Maintenance RC viewer, version 6.4.10.549752, issued October 1.
  • Project viewers:
    • Project Jelly project viewer (Jellydoll updates), version 6.4.10.549690, October 1.
    • Custom Key Mappings project viewer, version 6.4.5.544079, June 30.
    • Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
    • Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
    • Legacy Profiles viewer, version 6.3.2.530836, September 17, 2019. Covers the re-integration of Viewer Profiles.
    • 360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.

In Brief

  • Group chat issues are being worked on, but is pending the uplift work.
  • The long-promised increase in sound loop support from 10 seconds to 30seconds, first promised a fair while ago now, is … pending the uplift work – although it is rumoured (and subject to confirmation) to be a Premium Plus benefit.

2020 TPVD meeting week #42: summary

Isolation’s Passengers – blog post

The following notes are taken from the TPV Developer meeting held on Friday, October 16th, 2020. These meetings are generally held every other week, unless otherwise noted in any given summary. The embedded video is provided to Pantera – my thanks to her for recording and providing it. Time stamps are included with the notes will open the video at the point(s) where a specific topic is discussed. Note these summaries are not intended to be a full reporting on all topics discussed, but focus on those items that are more directly user-facing.

Another exceptionally brief meeting, with some discussion in chat, so please refer to the video as well.

SL Viewer News

[0:15-2:18]

As per my CCUG meeting notes, the Mesh Uploader RC was promoted this week to de facto release status – refer to the release notes (link below) for details of the cosmetic and informational changes made to the uploader (which are in part based on contributions by Beq Janus of the Firestorm team). This leaves the current official viewer release list as follows:

  • Current release viewer version 6.4.10.549686, formerly the Mesh Uploader RC released October t and promoted on October 14 – NEW.
  • Release channel cohorts:
    • Cachaça Maintenance RC viewer, version 6.4.10.549752, issued October 1.
  • Project viewers:
    • Project Jelly project viewer (Jellydoll updates), version 6.4.10.549690, October 1.
    • Custom Key Mappings project viewer, version 6.4.5.544079, June 30.
    • Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
    • Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
    • Legacy Profiles viewer, version 6.3.2.530836, September 17, 2019. Covers the re-integration of Viewer Profiles.
    • 360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.

General Viewer Notes

  • As per this week’s CCUG meeting, Vir is hopeful that the Legacy Profiles project viewer may be able to move forward soon using a workaround to get it past the current blocker of needing some web-side updates.
  • The Custom Key Mappings project viewer is being considered from promotion to RC status with its next update.
  • There is an upcoming RC viewer featuring some UI changes – possibly including the update to give HUD items in inventory their own inventory icon, as discussed by Steeltoe Linden in the CCUG week #40 meeting. Thus might be appearing in the next week or so.
The proposal UI change to distinguish HUD items from other objects in inventory.Left: how things are now, with HUDs using the same cube icon as other objects. Right: the proposed new HUD cog icon.
  • [5:24-6:54] There have been some issues with viewer documentation not appearing on the relevant web pages, leading to concerns that such documentation might be “going away”.
    • The problem has actually been caused by the mechanism used to generate the information not having been updated for some time (and also not even using current information), so when it was migrated to AWS services, it “fell apart”.
    • Rather than try to fix the issue, the decision has been made so that once uplift has been completed, the mechanism will be replaced with an easier to maintain process. However, it does mean that for the time being, some viewer related documentation may not be available.

In Brief

  • [9:20-10:00] BUG-228679: Media-on-a-Prim doesn’t register keystrokes correctly on Mac – unfortunately, no progress to report.
  • [12:12-13:40] A couple of TPVs have been holding off making a formal release of the Environment Enhancement Project (EEP) updates beyond their current beta releases in case LL make further EEP update releases like the recent Love Me Render #4 viewer release. However, the advice from the Lab to these TPVs is:
    • To consider the high-priority EEP fixes as being “done”, as the Graphics team is now focused on priority long-term architecture work (including moving the viewer beyond OpenGL).
    • That future EEP fixes, when made, will form a part of the regular Maintenance RC viewer releases, rather than having any dedicated RC viewer updates.
  • [13:49-16:58] EEP performance issues:
    • Some TPVs have reported significant performance / FPS drop-off with EEP (around 15%). However, the Lab is not seeing anything close to this with the official viewer.
    • Unless there are detailed reports filed on the issue – using the official release viewer – that provide information on where / when / how the issue is being witnessed (e.g. the kind of test carried out, where in-world, the hardware configuration and viewer settings used, the results obtained, etc.), that allow LL to reproduce the problems, they do not see what can be done by way of investigation / fixes.
      • It was pointed out that comparable testing on the official viewer is difficult due to the limited amount of video memory available (512 MB compared to a minimum of 1 GB on most TPVs), which also affects performance, and so might skew results / how results are seen (e.g. are they EEP or are they due to texture load issues?).
      • Vir indicated that the amount of assigned texture memory is something the Lab will be looking at in the future.

2020 CCUG meeting week #42 summary

La Vie, September 2020 – blog post

The following notes were taken from my audio recording and chat log of the Content Creation User Group (CCUG) meeting held on Thursday, October 15th 2020 at 13:00 SLT. These meetings are chaired by Vir Linden, and agenda notes, meeting SLurl, etc, are are available on the Content Creation User Group wiki page.

There was a lot of general discussion about animation synchronisation, cloud uplift (the majority of which I’ve already reported in my SUG and TPVD meetings of late), etc., none of which should out as really reportable.

SL Viewers

Default Viewer

The Mesh Uploader viewer, version 6.4.10.549686, was promoted to de facto release status on Wednesday, October 14th. This viewer provides additional information on mesh models prior to upload, with contributions from Beq Janus from the Firestorm team and also revisions by the Lab’s viewer team. Cosmetic changes with this viewer include:

  • Upgrade to preview resolution to 1024×1024.
  • Scalable preview window.
  • Fixed display of colours in preview.
  • Adjustment of colours to better correlate to in world (yellow frame of mesh, blue tint physics).
  • Rearrange UI elements to give more space for the preview even when not scaled up.

Informational changes  in this viewer include two new boxes: cost breakdown and physics breakdown – these provide access to information that has always been available to the viewer form the upload costs update message but were not previously shown to the user.

Legacy Profiles

The Legacy Profiles viewer (version 6.3.2.530836 at the time of writing) has been stuck awaiting a web-side update, but as the web team are all focused on Uplift work, attempts are being made to work around the delay in order to get the viewer moving again.

Graphics

  • Work has resumed at looking towards a replacement for OpenGL, particularly as a result of Apple’s plans to deprecate all support for it on their systems.
    • Vulkan had been an option under examination, and additional logging was added to the viewer to help identify how many users are on systems capable of running it.
    • It now turns out that a rather high number of users (up to a 1/3 of all users) are running Second Life systems that do not have support for Vulkan (e.g. those 5+years old, and notably systems with Intel integrated graphics).
    • As a result other options are also being looked at, as is the potential for supporting people on older hardware separately to those using systems capable of running with more recent graphics APIs – although this is an option the Lab would approach with caution, due to the added complexities in maintenance.
  • In addition, the graphics team is looking to see if the viewer UI rendering can be divorced from other aspects of viewer rendering to see if any performance improvements can be obtained by keeping the two separate – currently, a lot of cycles are taken up in the drawing of things like UI panels and buttons.

Jellydoll / ARCTan

  • Vir has been working on some updates to the Project Jelly viewer (version 6.4.10.549690 at the time of writing).
  • This viewer is a series of performance improvements related to Jellydoll avatars, and the additional work should be appearing in an update to the project viewer soon. After that, Vir is hoping to completely re-focus on the ARCTan work, which will eventually fold-in the Jellydoll work as well.
  • Should he be able to switch back to ARCTan, the work will remain split between avatar rendering (to be tackled first), and then in-world object rendering to follow along at some point after.
  • The work on avatar rendering will initially be focused on the data gathering aspect and better refining the avatar complexity calculations, followed by UI work to deliver more meaningful and consistent information to users about the rendering impact of their own avatar and those around them.

Date of Next Meeting

  • Thursday, October 29th, 2020.

2020 Simulator User Group week #42 summary

Souls of Dreams, September 2020 – blog post

The following notes were taken from the October 13th Simulator User Group meeting.

Simulator Deployments

Please refer to the server deployment thread for news and updates.

  • There was no deployment to the grid on Tuesday, October 13th.
  • On Wednesday, October 14th there should be an RC deployment, server maintenance release 550305, comprising further logging updates for the cloud migration work,together with additional logging that it is hoped will assist with locating the causes of teleport time-outs that have been occurring.

SL Viewer

There have been no updates to the current list of official viewers, leaving the pipelines as follows:

  • Current release viewer :Love Me Render #4 (EEP fixes), version 6.4.9.549455, released September 24, promoted September 28 – No Change.
  • Release channel cohorts:
    • Cachaça Maintenance RC viewer, version 6.4.10.549752, issued October 1.
    • Mesh uploader RC viewer, version 6.4.10.549686, October 1.
  • Project viewers:
    • Project Jelly project viewer (Jellydoll updates), version 6.4.10.549690, October 1.
    • Custom Key Mappings project viewer, version 6.4.5.544079, June 30.
    • Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
    • Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
    • Legacy Profiles viewer, version 6.3.2.530836, September 17, 2019. Covers the re-integration of Viewer Profiles.
    • 360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.

Cloud Uplift

  • While the Lab hasn’t as yet got to uplifting the “big three” simulator release candidate (RC) channels – BlueSteel,  LeTigre and Magnum – it was confirmed that as these are eventually migrated to AWS services, all things being equal, updates will continue to be deployed to them on Wednesdays.
  • People are reporting few or no region crossing issues when moving between adjacent regions that are running within the Lab’s co-lo facility and those now running on AWS.
  • The London City regions obtained permission from LL to confirm they are one of the private estates that has been assisting the Lab with cloud migration – see London City Uplifted.
  • Scripters confused about obtaining host names via scripted means as a result of the uplift work should refer to this forum thread.

Group Chat Issues

There is some confusion around the problems of group chat failures (in particular, although IMs can all suffer). these are routed in a September 1st Grid Status report that indicated the cause of the problem was known. Responding to requests made during the meeting for specifics on what the Lab had found, Oz Linden replied:

Sorry if that text got your hopes up. … We understand that that problem has a lot of impact on some people, which is why it’s on the status board. We’ve had some difficulty with getting the time to attack it given the demands of uplift. We have freed up some time, though, so we should start making progress on it soon.

As the current bout of group chat failures seems to be in part linked to teleports, there is a hope that the additional teleport-related logging being deployed to one (or more) of the RC channels on Wednesday, October 14th might provide some information on why this is the case.

In Brief

  • Some running services with external management to SL are still experiencing HTTP 499 connection loss errors. The request is that while the issue can be intermittent and irregular, if those experiencing could file a Jira specifying time, location, and object ID when they see the issue, it would be helpful.

October 2020 WUG summary with SL Mobile news

The Web User Group meeting venue, Denby

The following notes are taken from my recording of the Web User Group (WUG) meeting, held on Wednesday, October 7th, 2020. These meetings are generally held on the first Wednesday of the month, with dates and venue details available via the Web User Group wiki page.

When reading these notes, please keep in mind:

  • This is not intended as a chronological transcript of the meeting. Items are drawn together by topic, although they may have been discussed at different points in the meeting.
  • Similarly, and if included, any audio extracts appearing in these summaries are presented by topic heading, rather than any chronological order in which they may have been raised during the meeting (e.g. if “topic X” is mentioned early in a meeting and then again half-way through a meeting, any audio comments related to that topic that might be included in these reports will be concatenated into a single audio extract.

Web Properties Updates

The primary focus remains on uplift work, and while there have been updates to some of the user-facing web properties and image updates for web pages to reflect the season, and some new pages for events such as the September Titmouse event (see: Second Life to have a smashing time with Titmouse) there was not much to report.

There should be more news on web updates available at the next WUG meeting in November.

General Uplift Notes

See also my recent TPV Developer meeting and CCUG meeting notes.

In general, the Lab’s aim is to have all uplift work completed by the year-end.

  • This is getting all services and properties running via cloud services without additional work on making improvements (unless it makes specific sense to make changes as a part of the uplift process, as we saw with region crossings recently).
  • Improvements will follow-on from the initial uplift work as the Lab gains greater familiarity with running services in the cloud.
  • Similarly, there are no plans to offer new land products or alter the capabilities of the existing region types on completion of the uplift (although it has been indicated elsewhere these may come over time in the future).
  • There will obviously be a resumption in region availability once the uplift work has been completed.
  • The end-of-year target is a fairly “hard” target, with the caveat that the unexpected can happen to delay things – although that doesn’t appear to have happened thus far.

Mobile Client

iOS

  • The iOS version of the mobile client was submitted to Apple for review – and was rejected. LL is currently working to address the issues as a part of Apple’s review.
  • Once this has been done, the app will again go through LL’s QA process and then be re-submitted to Apple for evaluation / App Store listing,
  • The version currently remains available to those on the closed Alpha only.
  • Interestingly, the Lab’s data suggests to them that the number of users with iOS is slightly greater than those with Android, although it’s not clear had this data is derived.

Android

  • The android version is still being developed, but remains “some way” behind the iOS version.
  • Currently it is not ready for any user-based testing.

General Points of Note

Please also refer to my periodic SL Mobile updates, which are made as and when there are significant updates in the status of the client.)

  • It is initially intended to be a communications app, aimed at allowing those who are away from their their usual means of access Second Life to communicate with friends via IM.
  • This means that the initial release will not have capabilities such as world rendering, support for making sales / purchases through it, etc.
  • However, the client will be enhanced over time.
    • Group and group chat functionally were specifically mentioned, so might be the first set of enhancements.
    • Inventory management was also mentioned, as is apparently being considered, but is described as having its own “own bucket or twelve of worms”, so no decision on if / when it may appear has as yet been made.

As an aside, for those who point to the likes of Lumiya and its capabilities and ask why the LL client isn’t offering the same right from launch, it’s worth remembering that Lumiya took several years to develop its capabilities  iteratively – it wasn’t launched with all of its features fully-fledged.  This is the route the Lab is trying to take with their mobile client.

Marketplace

Demos

  • In general, demo version of items on the Marketplace should be linked to the full version.
  • There are reports that since making demo items filterable in search, there has been an increase in unlinked demo items
  • It’s not clear if this is because people are trying to avoid the filter, or that incoming new merchants simply don’t understand they can link demos to full versions, or even if it is a functional limitation of the filter to pick-up all demo items.
  • The Lab will take a look at this, and ideas on how to improve the information given on creating listing are requested via feature request.

Suggestions

A number of suggestions for the MP were put forward at the meeting:

  • Add a feature similar to see on the old XStreetSL marketplace: a request for purchasers of items to provide a review.
  • Provide a means by which creators can enable a report their last log-in date on their stores / listings. The idea here is to increase the confidence of purchasers that they will receive after-sales support, etc., should it be required.
  • An option for a “Show Only # Star Reviews” (where # is the number of stars) and / or to show a summary of star reviews by #of stars (as with Amazon product listings).
  • LL have requested these all be submitted as feature requests.

Last Names

  • Last Names are still seeing a good response from Premium users.
  • LL might opt to offer a new selection of names to reflect the time of year – but again, this would be dependent upon the current volume of uplift work.
  • Currently, the Lab has around 10,277 names that could potentially be used over time.

Next Meeting

  • Wednesday, November 4th, 14:00 SLT.