The following notes were taken from the Tuesday, December 15th Simulator User Group (SUG) meeting.
Server Updates
Deployments in week #50 got a little confusing, with both SLS (Main) and some RC (Magnum) RC channel servers updated to release 553685, with the remainder of the simulators being updated to the same release on Tuesday, December 15th. At the SUG, Rider linden explained the situation thus:
So let’s see. We ended up having one extra set of rolls on Friday and then again today to catch the last of the stragglers. It was basically just an internal configuration change to help support uplift. I’m confident that that was the final roll of the year.
SL Viewer
The Dawa maintenance viewer updated to version 6.4.12.553723 on December 15th. The rest of the current crop of official viewers, remain as follows:
Current release viewer version 6.4.11.551711, formerly Cachaça Maintenance RC viewer promoted on November 12 – No Change.
Release channel cohorts:.
Custom Key Mappings project viewer, version 6.4.12.552100, November 12.
Project viewers:
Project Jelly project viewer (Jellydoll updates), version 6.4.12.552224, December 4.
Simple Cache project viewer, version 6.4.11.551403, issued on November 12.
Legacy Profiles viewer, version 6.4.11.550519, October 26.
Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.
In Brief
Map tiles are still not being regularity updated, however the Lab both identified the cause of the issue and a possible fix, however, it now seems probable this will not be deployed until after the holiday break.
The updates to Group chat made over the last day or so should not be seen as a “complete fix” for issues. Rather, it is intended to hopefully improve things for at least some who have experienced problems,with further work to come, with Oz Linden noting:
There is no doubt that group chat needs additional work. We have recently resolved some problems that we were able to diagnose, but more work will be needed as soon as we can devote some time to it.
It is possible that post-Uplift, EEP might see some further work, including an adjustment to the default Mainland ambient lighting.
The Legend of Wooley Swamp, October 2020 – blog post
The following notes are taken from the TPV Developer meeting held on Friday, December 11th, 2020.
These meetings are generally held every other week, unless otherwise noted in any given summary.
The embedded video is provided to Pantera – my thanks to her for recording and providing it. Time stamps are included with the notes will open the video at the point(s) where a specific topic is discussed.
Note these summaries are not intended to be a full reporting on all topics discussed, but focus on those items that are more directly user-facing.
Another exceptionally brief meeting, with some user-user discussion in chat at the end (post-meeting) on Open Metaverse, so please refer to the video as well.
There have been no updates to the current group of official viewers in the pipelines, leaving things as:
Current release viewer version 6.4.11.551711, formerly Cachaça Maintenance RC viewer promoted on November 12 – No Change.
Release channel cohorts:
Dawa Maintenance RC Viewer, version 6.4.12.553053, released December 3rd.
Custom Key Mappings project viewer, version 6.4.12.552100, November 12.
Project viewers:
Project Jelly project viewer (Jellydoll updates), version 6.4.12.552224, December 4.
Simple Cache project viewer, version 6.4.11.551403, issued on November 12.
Legacy Profiles viewer, version 6.4.11.550519, October 26.
Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.
General Viewer Notes
Both the Key Mappings and Dawa Maintenance RC viewers are approaching the point where either one could be considered ready from promotion to de facto release status. However, give the low crash rate and high stability data coming from the current release viewer, is is possible that neither one of these RCs will be promoted before the new year.
Two more RC viewers should be appearing shortly:
The Simple Caching viewer that replaces the current VFS cache with a much simpler mechanism. This is currently in review for promotion as an RC viewer.
The current Jellydolls project viewer is also approaching the point of being promoted to RC status.
[7:03-7:53] There are a number of EEP issues that have been flagged as “priority”, but its not clear if these will be addressed in the next Love Me Render (LMR) RC viewer or not (due to voice break-up as Vir was discussing).
There is an expectation within LL that any updates to the rendering system (including what are regarded as “critical EEP fixes” by some users) will be condemned as further “breakage” in EEP.
Thanksgiving Bakefail: LL believe they now have a fix for this issue, which is awaiting deployment.
BUG-229824: “[EEP] Cannot apply any EEP setting to ANY parcel or region:” reported on December 11th, a fix for this issues has been deployed, which should correct it. HOWEVER, any settings assets uploaded between 14:40 SLT on Thursday, December 10th and and 11:50 SLT on Friday December 11th, will not work. They will need to be deleted and replaced.
[5:24-6:23] The Lab is “fast closing in”on having all SL services transitioned to operating via AWS. The percentage of remaining systems to be migrated in being put at around 3%.
Again, please note that this is only the migration work; there will be further work to come with regards to fine tuning, performance updates, etc. For now the focus is on gathering statistics on how simulators, systems and services are all running.
[8:29-13:50] general discussion in chat and voice on how best to manage user / creator expectations on what may “break” or be altered as a result of upcoming work on the viewer’s rendering system.
There may well be “breakages” / changes because an overhaul of the rendering system is unlikely to try to achieve a “like for like” swapping of issues between one rendering pipe and another, but to try to improve things as and where possible; as such, so “breakages” are perhaps inevitable.
One problem here is the number of workarounds that have been employed over the years in order to circumvent perceived / actual issues within the current rendering system, and which may not continue to work as expected as changes are made.
Another issue is when to try to inform users and creators – the rendering system work is liable to be long-term, and made not yield immediately changes; ergo, providing notification too early could lead to a lot of upset and concern, only for it to evaporate and the warning of changes to be forgotten, if the changes are not immediately implemented.
Further to this is just how to inform the majority – many users do not read official blog posts and forums (and even when news is pushed through third-party blogs, such as this one, it still only tends to reach a narrow audience). The viewer message of the day also has limited impact, simply because the official viewer is not the most widely used viewer in SL.
Plus, even with forewarning, the natural course of things is for people to keep using with which they are familiar right up to the point where changes are made.
However, once the Lab has a better idea of what subsets of rendering elements are likely to change, efforts will be made to communicate these to users in advance.
[13:55-16:25] Another change with the viewer is the on-going removal of code that is not longer being actively used. A further example of this is that there are two protocols used for handling requests for logging-in to SL. One of these (XML-RPC) is almost never used nowadays, so this code may well be removed – with warning beforehand – in the future.
The week sees a series of deployments between Monday and Wednesday:
On Monday, December 7th, the simulators on the SLS Main channel were updated to simulator release 553168 contacting “some internal tweaks”.
On Tuesday, December 8th, the RC channels were updated to simulator release 553176, which again contains “some internal tweaks”.
If all goes will with the RC deployment, 553176 will be deployed to the SLS Main channel on Wednesday, December 9th.
Commenting on the week’s deployment, Rider Linden said:
So. We’re just about done with the tight sequence of rolls. Tomorrow [Wednesday 8th] we should be doing the main channel again, and then all will be right with the world. We are hoping that that is the final roll of the year. In January we should be back onto our regular Tuesday/Wednesday schedule. We know that the packed schedule has been disruptive over the last week, so thank you for being patient with us.
While there are no planned deployments between the end of this week and year-end, it is likely all regions will be restarted before the holiday break at the Lab to “keep things fresh”.
SL Viewer
The Start of the week has seen no change to the current crop of official viewers, leaving them as follows:
Current release viewer version 6.4.11.551711, formerly Cachaça Maintenance RC viewer promoted on November 12 – No Change.
Release channel cohorts:
Dawa Maintenance RC Viewer, version 6.4.12.553053, released December 3.
Custom Key Mappings project viewer, version 6.4.12.552100, November 12.
Project viewers:
Project Jelly project viewer (Jellydoll updates), version 6.4.12.552224, December 4.
Simple Cache project viewer, version 6.4.11.551403, issued on November 12.
Legacy Profiles viewer, version 6.4.11.550519, October 26.
Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.
In Brief
Map tiles are still not being regularity updated, however the Lab both identified the cause of the issue and a possible fix, however, it now seems probable this will not be deployed until after the holiday break.
LL have suggested that the Map system might be looked at in the future, with a request being made as to what people would like to see, were such a project to go ahead. Some of the suggested made at the meeting included:
The ability to add a logo over regions, rather than having to make one from prims (allowing for constraints that would be required to prevent abuse (such as having an expanded set of iconography?).
Allow the map to ignore region surrounds so the terrain can be seen on the map, rather than the map tile being a blank coloured square.
Heat maps showing traffic over varying time periods, a-la the old Metabolt client.
Better zoom capabilities.
Indicators / textures for showing whether the region is currently in daylight or night-time.
Work is continuing to try to improve the teleport issues some are experiencing.
The following notes were taken from my audio recording and chat log of the Content Creation User Group (CCUG) meeting held on Thursday, December 3rd 2020 at 13:00 SLT. These meetings are chaired by Vir Linden, with dates available via the SL Public Calendar.
A new Maintenance RC viewer, version 6.4.12.553053, was release on Thursday, December 3rd. Otherwise, the current selection of official viewers remains unchanged:
Current release viewer version 6.4.11.551711, formerly Cachaça Maintenance RC viewer promoted on November 12 – No change..
Release channel cohorts:
Custom Key Mappings project viewer, version 6.4.12.552100, November 12.
Project viewers:
Simple Cache project viewer, version 6.4.11.551403, issued on November 12.
Project Jelly project viewer (Jellydoll updates), version 6.4.11.551213, November 2.
Legacy Profiles viewer, version 6.4.11.550519, October 26.
Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.
ARCTan / Jellydoll / Imposter Avatars
Vir has most recently looking at timers / frame rate variability.
As a reminder:
The current ARCTan work is focused on the viewer-side updates to avatar complexity calculations.
Work on providing in-world object rendering costs (LOD models, etc.) which might affect Land Impact will be handled as a later tranche of project work, after the avatar work.
Graphics
Work has commenced on separating out the rendering of viewer UI elements from scene rendering.
The reason for this is that the UI currently gets rendered on per frame basis, which is performance expensive on what is essentially a relatively static element of the viewer.
The aim is to revise the UI such that it is only re-drawn was needed, and it is hoped this will result in a performance gain in terms of viewer frame rates.
When this work is in a condition where it can be released, it will appear in its own project / RC viewer, and not as a part of the Love Me Render graphics-specific viewer pipeline.
There may be an issue with the viewer ignoring avatar custom joint positions if they do not have a weight associated with them. This appears true for .DAE files exports from Maya, and possibly with exports from Blender 1.7.9.a / 1.7.9.b (although Avastar may automatically compensate for unweighted custom joints by given them a default weighting). However, it is not clear if the issue lies with the modelling software or the mesh uploader, and its been requested that those experiencing these issues check the mesh upload data to see if anything is being pruned.
The following notes were taken from the Tuesday, December 1st Simulator User Group (SUG) meeting.
Server Updates
On Monday, November 30th, the simulators on the SLS Main channel were all updated to version 552571, matching the RC channels.
Originally, it had been planned to follow this with a tight series of rolls over the next few days, with simulators updated in batches at roughly 16-hour intervals. However, at the time of writing this is on hold, as Rider Linden explained:
That plan, like so many others, is now up in the air. There are a couple of last minute snags that we ran into and so we’re evaluating what the best way to do these deploys is going to be and we do not yet have an answer. We will keep you all informed on the blog where things settle out.
This means that – again at the time of writing – the rolls listed on the grid status pages are TBD, and subject to update from LL.
No Change Window
Normally, December sees a limited number of deployments for the simulators, with a “no change” window coming into effect over the holiday period (usually from just before the Christmas break to just after New Year’s day).
However this year, thanks to the migration work, this are apparently more fluid, and dates – if any – have yet to be announced.
SL Viewer
The Start of the week has seen no change to the current crop of official viewers, leaving them as follows:
Current release viewer version 6.4.11.551711, formerly Cachaça Maintenance RC viewer promoted on November 12 – No Change.
Release channel cohorts:
Custom Key Mappings project viewer, version 6.4.12.552100, November 12.
Project viewers:
Simple Cache project viewer, version 6.4.11.551403, issued on November 12.
Project Jelly project viewer (Jellydoll updates), version 6.4.11.551213, November 2.
Legacy Profiles viewer, version 6.4.11.550519, October 26.
Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.
We know what’s going on but unfortunately it’s going to take a bit of time before we can have a fix. Some of the other work on getting to the cloud needs to get done first. It’s complicated by how inter-dependent our systems are.
On Tuesday November 24th, RC channels were updated to simulator version 552571, comprising “some bug fixes and internal tweaks”. However, one fix should correct the inability to correctly set permissions (e.g. edit rights) for friends.
There is no planned SLS Main channel deployment for week #48.
Week #49
If all goes to plan at the Lab, week #49 (commencing Monday, November 30th) will see a daily series of rolling restarts across the grid. Due to start on Monday, this will be batches of regions being restarted, with the plan that restarts are staggered to avoid too much disruption, and if everything proceeds smoothly, each region should only be restarted once at some point in the week.
Commenting on the restarts, Maxidox Linden stated:
I think it will be something like at most 14-16 hours between rolls. Though as Rider says unless something changes we don’t plan to roll the same region in two successive rolls. And we’re going to do our best to avoid times with tons of people on-line if we can.
These deployments are liable to occur at 16-hour intervals.
Uplift Status
As per my blog post from week #48 (see: LL confirms Second Life regions now all on AWS), whilst all regions are now running on AWS services, the work in transitioning all of the Second Life back-end services is not complete, and LL are still “operating with one foot in either camp” – and this may be exacerbating the problems currently being experienced by some.
Another factor could be the different communications routes between viewers and servers following the move from operating out of the Lab’s co-lo facility in Arizona to the Amazon centre in Oregon. For some, this has definitely resulted in a noticeable increase in basic ping times to / from the servers, although for others, this has barely changed.
Commenting on the general state of play, Mazidox Linden observed:
We are not yet at what I would call “Final uplift performance” (that is to say, without any explicit attempts at tuning performance and behaviour of system communication). There is still plenty of stuff making that round trip over hundreds of miles to the data centre, slowing things down.
In this respect, some of the issues people are noticing at the moment may be down to the fact that LL haven’t as yet started fine tuning things, and are unlikely to do so until all services are running via AWS. As such, some of the problems people are noting might be down to this. In this respect, Oz Linden noted:
We’re much more focused right now on getting things other than the simulators uplifted and fixing anything that really breaks. Performance problems are a step down in priority until that’s done, but we won’t forget about them.
SL Viewer
The Start of the week has seen no change to the current crop of official viewers, leaving the as follows:
Current release viewer version 6.4.11.551711, formerly Cachaça Maintenance RC viewer promoted on November 12 – No change
Release channel cohorts:
Custom Key Mappings project viewer, version 6.4.12.552100, November 12.
Project viewers:
Simple Cache project viewer, version 6.4.11.551403, issued on November 12.
Project Jelly project viewer (Jellydoll updates), version 6.4.11.551213, November 2.
Legacy Profiles viewer, version 6.4.11.550519, October 26.
Copy / Paste viewer, version 6.3.5.533365, December 9, 2019.
Project Muscadine (Animesh follow-on) project viewer, version 6.4.0.532999, November 22, 2019.
360 Snapshot project viewer, version 6.2.4.529111, July 16, 2019.
In Brief
Group Chat: There has been an update to the Group chat servers which the Lab hopes will help alleviate the issues of the last couple of months. The hope is that as a result of this, things should be somewhat better, although it is acknowledged things are “not perfectly solid” as yet.
Map Tiles: there is a known issue with in-world Map tiles failing to update. At the time of writing, there was no ETA on when a fix will be implemented.
Teleports: people are still reporting teleport failures, although data collected by the Lab using additional logging apparently shows the overall level of teleport failures as being back to “normal” after the recent spike.
Textures: Slow texture loads: people are reporting slower than usual texture loading. When isn’t currently understood (given textures are among the data coming to users via the CDN, so not directly a part of the AWS transition). The speculation offered by Oz Linden is that where texture messaging is concerned, it may not be going as fast as LL like.
Scripts: there have also bee reports of some regions initially showing improved script performance, only to apparently drop back to “pre-uplift” levels of processing. Commenting on this, Maxidox Linden stated:
If you’re seeing changes to scripts run the likely explanation is that there is contention for shared system resources beneath the simulator layer. That is something we had only mild control over before and have even less control over now.. That said, it’s on our radar. I can’t promise that even when someone gets time to look at it there will be anything we can *do*, but we are aware, and we’re not ignoring it. …
I mean, it is almost possible that we’re calculating that number wrong Lucia, because we have certainly changed the hardware the simulators are running on in ways that the people who made that statistic probably never imagined. I’m not going to swear that is or isn’t happening, but it will certainly be one of the many things we look at.
Inara Pey: Living in a Modemworld 