Category: Simulator User Group

The following notes were taken from the Tuesday, November 16th, 2021 Simulator User Group (SUG) meeting. The meeting was recorded by Pantera Północy, and the video is embedded at the end of this summary. Note this summary focuses on the key points of the meeting; where there is something to report, the video should be referred to should full details of the meeting wish to be reviewed.

Server Deployments

• The are no planned deployments to the majority of the region simhosts.
• On Wednesday, November 17th, the servers on the Bluesteel channel should be updated with a fresh version of the tools update simulator version.
• It is unlikely the simulator version on Bluesteel will be rolled to additional simhosts / channels prior to US Thanksgiving, as there are some issued the Lab has noted, which are proving a little hard for the Lab to track down in testing on Aditi and the Lab’s internal test grid.
• It is likely that the number of server deployments between now as the end of the year will be “minimal”.

SL Viewer

This list reflects those official viewers available via Linden Lab.

• Release viewer: version version 6.5.0.565607, formerly the Maintenance RC and dated November 10, promoted November 15 – this viewer now contains a fix for the media issues caused by the Apple Notarisation viewer.
• Release channel cohorts (please see my notes on manually installing RC viewer versions if you wish to install any release candidate(s) yourself):
  • The Tracy Integration RC viewer version 6.4.23.563771 (dated Friday, November 5) issued Tuesday, November 9.
  • 360 Snapshot RC viewer, version 6.5.0.564863, issued October 21.
  • Simplified Cache RC viewer, version 6.4.23.562623, dated September 17, issued September 20.
• Project viewers:
  • Performance Improvements project viewer updated to version 6.4.24.565324 (dated November 5) November 9.
  • Performance Floater project viewer, version 6.4.23.562625, issued September 2.
  • Mesh Optimizer project viewer, version 6.4.23.562614, issued September 1.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

In Brief

• It has been reported that general region performance is better with the new toolset simulator version. However, when something is rezzed with scripts there is a 2 second rez delay if there are child prims in the object – if it is a singular object, there’s hardly any delay. The object appears inworld but on_rez() and object_rez() don’t trigger for a solid 2 seconds. This does not happen for a single prim object. It’s not clear if this is some variant of BUG-228939; however, that tended to become more noticeable the longer a region was running, but this issue seems to occur immediately after a region is restarted. See here for more.
• User Fourmilab has also been investigating the user of the broader “2-second delay”, and has posted findings here and here.
• BUG-231303 “Scripted agents can no longer log in” is still under investigation, although it had been hoped the issue might have been resolved by now. The issue has caused some “alarm” (on behalf of those doing so) that so much is being run on what is effectively outdated “abandonware”.
• Work is going on to make Map updates more reliable, and it is possible there may be further updates to the Map system in 2022.

 

The following notes were taken from the Tuesday, November 9th, 2021 Simulator User Group (SUG) meeting. The meeting was recorded by Pantera Północy, and the video is embedded at the end of this summary. Note this summary focuses on the key points of the meeting; where there is something to report, the video should be referred to should full details of the meeting wish to be reviewed.

Server Deployments

• Tuesday, November 9th saw simhosts on the SLS Main channel restarted, but no actual simulator update, leaving them running version 565008.
• Wednesday, November 10th should see regions on the RC channel restart:
  • The majority of RC simhosts will be restarted without any update, also leaving them running simulator version 565008.
  •  Those servers on the Bluesteel channel will be updated to version 565204, containing the outcome of the recent work in updating the simulator toolsets. Testing of the simulators using the new tools has shown “significant improvement” in simulator operations, so the hope is that this will be reflected in the deployed version.

SL Viewer

This list reflects those official viewers available via Linden Lab.

• Release viewer: version version 6.4.23.564172, formerly the Apple Notarisation Fix RC viewer, issued September 24 and promoted October 15.
• Release channel cohorts (please see my notes on manually installing RC viewer versions if you wish to install any release candidate(s) yourself):
  • 360 Snapshot RC viewer, version 6.5.0.564863, issued October 21.
  • Maintenance RC viewer updated to version 6.5.0.564805, on October 20.
  • Simplified Cache RC viewer, version 6.4.23.562623, dated September 17, issued September 20.
• Project viewers:
  • Performance Improvements project viewer, version 6.4.23.564530, dated October 12.
  • Performance Floater project viewer, version 6.4.23.562625, issued September 2.
  • Mesh Optimizer project viewer, version 6.4.23.562614, issued September 1.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

Apple Notarisation Viewer Issue

With the release of the Apple Notarisation Viewer there were updates to many of the viewer’s third party libraries, and some of these updates have be found to cause issues related to playback of certain media types in-world (notably MP3s and MP4s). A fix is in progress, and once ready, LL intend to fast track it through QA ahead of other viewer updates and make an RC viewer with the fix available ASAP.

TLS Changes

Monty Linden provided an update on the status of disabling TLS 1.0 / 1.1 and future certification work:

There are some ongoing issues [with] a combination of effects in play. In Bug-231303, we have reports of failed cipher negotiation. Right now, only three ciphers are available under tls1.2 on login. I hope to strictly expand that. *HOWEVER* this is where that old viewer cert checking code comes in and does the wrong thing. Future certs will not have the ‘key encipherment’ usage extension – and those old checks need to disappear from all viewers and bots and comms libraries. 

We can’t check every combination people try to use. We can’t even check many combinations, but we’re heading in the direction of wider conformity of the rest of the net. I would like to be able to offer testing on Aditi prior to future changes but we’ve coupled the two grids in an unfortunately way. That might take time to separate. Maintained clients that updated the cert check should work. Old libomv stuff that is broken currently may or may not be repaired (some will, some won’t). Anyone truly doing their own thing will have to answer to the crypto gods.

The reg changes should hold up. But more changes will be coming in ’22 and beyond. Things that have gone unmaintained for years are going to be very susceptible to crypto changes in the wider world. This is the warning.

FYI, consider the 2020q1 profile in this as a target.

For client authentication where Linden doesn’t control the server endpoint, I’m thinking other authentication schemes are in order. OAuth2, etc. We had a high-level LSL discussion the other week. Better supporting auth schemes for scripting would be one of these. But it also enters into having private storage and secrets management. I.e. all the stuff real cloud providers have to do. Even that is a project. All of which would be A Really Good Thing.

Monty Linden

This commentary rolled into a wider discussion of potential scripting options and of further HTTP work. In terms of the former, Monty indicated that the Lab is not ready to increase script memory limits, because there is a believe that ” we can make the memory you have do more.” He also indicated that amongst the backlog of work is a Mono update, but could not say when that might progress to a point of being worked on.

The following notes were taken from the Tuesday, November 2nd, 2021 Simulator User Group (SUG) meeting. The meeting was recorded by Pantera Północy, and the video is embedded at the end of this summary. Note this summary focuses on the key points of the meeting; where there is something to report, the video should be referred to should full details of the meeting wish to be reviewed.

Server Deployments

• Tuesday, November 2nd saw simhost on the SLS Main channel updated to the simulator release deployed to the RC channel in week #43. This includes a revised implementation of PRIM_PROJECTION.
  • For the time being, it will be write only, meaning it can used in llSetPrimitiveParams but not in llGetPP. The associated wiki documentation has yet to be updated.
  • This update will also include BUG-231158 Allow llGetNotecardLine to return more than 255 bytes (to a maximum of 1023 characters).
• Wednesday, November 3rd should see regions on the RC channel restart, but no deployment made, and version numbers should not change.

SL Viewer

This list reflects those viewers available via the first four links in the LL Viewer Resources section, below.

• Release viewer: version version 6.4.23.564172, formerly the Apple Notarisation Fix RC viewer, issued September 24 and promoted October 15.
• Release channel cohorts (please see my notes on manually installing RC viewer versions if you wish to install any release candidate(s) yourself):
  • 360 Snapshot RC viewer, version 6.5.0.564863, issued October 21.
  • Maintenance RC viewer updated to version 6.5.0.564805, on October 20.
  • Simplified Cache RC viewer, version 6.4.23.562623, dated September 17.
• Project viewers:
  • Performance Improvements project viewer, version 6.4.23.564530, dated October 12.
  • Performance Floater project viewer, version 6.4.23.562625, issued September 2.
  • Mesh Optimizer project viewer, version 6.4.23.562614, issued September 1.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

Apple Notarisation Viewer Issue

With the release of the Apple Notarisation Viewer there were updates to many of the viewer’s third party libraries, and some of these updates have be found to cause issues related to playback of certain media types in-world (notably MP3s and MP4s). A fix is in progress, and once ready, LL intend to fast track it through QA ahead of other viewer updates and make an RC viewer with the fix available ASAP.

TLS Changes

As per the announcement by April Linden on October 8th, Linden Lab turned off support for Transport  Layer  Security (TLS) 1.0 and TLS 1.1  security protocols on all log-in services (in line with the Internet Engineering Task Force (IETF) having officially deprecated both protocols in March 2021). However, it resulted for older services running Windows 7 (lacking TLS 1.2 support unless running with Service Pack 1 – see BUG-231303 “Scripted agents can no longer log in”, and also this blog post from Cinder Roxley.

In Brief

• The Map tile server is currently “note working”, so the Map may display issues in updating. Steps are being taken to identify tiles that are not updating and rectifying the problem – it is hoped the fix should be in place by November 3rd.
• Tool update work / server OS update remains the primary focus within the simulator engineering team.
• There was general discussion on possible work the team might try around temp attachments and alternatives to HUDs, but nothing that is close to being discussed in depth.

The following notes were taken from the Tuesday, October 26th, 2021 Simulator User Group (SUG) meeting. The meeting was recorded by Pantera Północy, and the video is embedded at the end of this summary. Note this summary focuses on the key points of the meeting; where there is something to report, the video should be referred to should full details of the meeting wish to be reviewed.

Server Deployments

• Tuesday, October 26th saw the simhosts on the SLS main channel restarted without any update.
• Wednesday, October 27th should see the deployment of  simulator version that includes a revised implementation of PRIM_PROJECTION.
  • For the time being, it will be write only, meaning it can used in llSetPrimitiveParams but not in llGetPP. The associated wiki documentation has yet to be updated.
  • This update will also include BUG-231158 Allow llGetNotecardLine to return more than 255 bytes (to a maximum of 1023 characters).

SL Viewer

There have been no updates to the current crop on official viewers to mark the start of the week, leaving the pipelines as:

• Release viewer: version version 6.4.23.564172, formerly the Apple Notarisation Fix RC viewer, issued September 24 and promoted October 15 – see notes below.
• Release channel cohorts (please see my notes on manually installing RC viewer versions if you wish to install any release candidate(s) yourself):
  • 360 Snapshot RC viewer, version 6.5.0.564863, issued October 21.
  • Maintenance RC viewer updated to version 6.5.0.564805, on October 20.
  • Simplified Cache RC viewer, version 6.4.23.562623, dated September 17, issued September 20.
• Project viewers:
  • Performance Improvements project viewer, version 6.4.23.564530, dated October 12.
  • Performance Floater project viewer, version 6.4.23.562625, issued September 2.
  • Mesh Optimizer project viewer, version 6.4.23.562614, issued September 1.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

Apple Notarisation Viewer Issue

With the release of the Apple Notarisation Viewer there were updates to many of the viewer’s third party libraries, and some of these updates have be found to cause issues related to playback of certain media types in-world including, but not possibly limited to MP3s and MP4s. LL are working towards a fix, but in the meantime, workaround are offered:

• Windows:
  • Uninstall the Second Life viewer the usual way.
  • Navigate to your Program Files folder (Win 64-bit) or Program Files (x86) (Win 32-bit); locate and delete the “SecondLifeViewer” folder.
  • Download and install the Simplified Cache viewer (the previous release viewer).
• Apple Mac:
  • Remove the Apple Notarisation viewer from your system.
  • Download and install the Simplified Cache viewer (the previous release viewer).
  • Read the instructions on this page to work through any occurrences of unwanted notarisation warnings.

In Brief

• With the on-going work to analyse viewer performance using the Tracy debugger / system analyser (see my TPV Developer summaries for more), the simulator team are looking to try and leverage that work on the simulator side, once the current tools upgrade on the server-side has been completed.
• There was a 10-minutes discussion around animated particles / revising the particle systems, but there is nothing on the card for immediate work.
• The upcoming simhost operating system upgrade (also to follow the completion of the tools upgrade) was also discussed. Essentially there is a lot of work to be done, and the potential for regressions or other issues to occur. However, LL are planning things as carefully as possible; no overall decision as to how it will be deployed to the main grid come time for it to do so – mainly because the work has yet to start.

The following notes were taken from the Tuesday, October 19th, 2021 Simulator User Group (SUG) meeting. The meeting was recorded by Pantera Północy, and the video is embedded at the end of this summary. Note this summary focuses on the key points of the meeting; where there is something to report, the video should be referred to should full details of the meeting wish to be reviewed.

Server Deployments

See the current server deployment thread for updates

• Tuesday, October 19th saw the simhosts on the SLS main channel restarted without any update. for some this may have seen the server release number increment to 564759. However, there were some issues, so some simulators on the channel were refreshed with release 564394.
  • LL is working to get to a state where version numbers only increment when there is a deployment and rolling restart, rather than incrementing as a result of a rolling restart alone.
• Wednesday, October 20th should see the deployment of  simulator version 564610. This includes the following feature requests:
  • SL-15994 Add “force” option to estate_environment command (non-public Jira).
  • BUG-231158 Allow llGetNotecardLine to return more than 255 bytes.
  • BUG-229639 [FEATURE REQUEST] PRIM_PROJECTION.

Operating System Upgrade

As I’ve recently noted, following the current server tools update, the Lab plans to update the underpinning operating system. This work is now on the horizon. Or, to put in in the words of Mazidox Linden:

Do you all remember back in the Before Times when we were still in the colo and offices were still a thing and we did simulator operating system upgrades? Well, we’re in the cloud, I’m still working from a cramped desk in a corner of my guest room, but guess what we’re doing!

I know an OS upgrade is not the most attractive thing nor fun to hear about, so I’m willing to offer you a carrot. We’ve seen at least 10% more scripts running per frame on the same hardware with the same starting state for a simulator. [SO] the sooner we get this os upgrade tested the sooner it gets out the door and into your hands.

Those wishing to test their scripts, etc., on the new OS version can currently do so on the following regions on Aditi (the beta grid): Cale, Gothlauth, Jigglypuff, Leafeon, London City, London City Gateway, Mauve, Moonberry, Riccione, Sapas, Smithereens, Snark, Testylvania Sandbox, and Woods of Heaven.

Mazidox also stated:

If you have a region on Agni that has some significant script load, or unusual characteristics like lots of physics utilization, pathfinding… Regions that really stress aspects of the server without people being on them, please feel free to IM me here on Agni and ask for the region to be copied to Aditi on a first come first serve basis.

SL Viewer

There have been no updates to the current crop on official viewers to mark the start of the week, leaving the pipelines as:

• Release viewer: version version 6.4.23.564172, formerly the Apple Notarisation Fix RC viewer, issued September 24 and promoted October 15.
• Release channel cohorts (please see my notes on manually installing RC viewer versions if you wish to install any release candidate(s) yourself):
  • Maintenance RC viewer updated to version 6.4.23.564063, on September 21.
  • Simplified Cache RC viewer, version 6.4.23.562623, dated September 17, issued September 20.
• Project viewers:
  • Performance Improvements project viewer, version 6.4.23.564530, dated October 12.
  • 360 Snapshot project viewer, version 6.4.23.563579, issued September 3.
  • Performance Floater project viewer, version 6.4.23.562625, issued September 2.
  • Mesh Optimizer project viewer, version 6.4.23.562614, issued September 1.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

In Brief

• There has been a lot of discussion about the viewer’s bandwidth slider on the forums again recently, which has also touched on default cache sizes
  • As noted in that thread, the bandwidth slider in particular is less useful nowadays than once was the case, prompting the question on whether or not it should be retained within the viewer.
  • During the meeting, the question was asked if the simulators are liable to become defensive if the setting were to be fixed (presumably relatively high) within a viewer like Firestorm, which could provoke “microburst contention” due to the volume of UDP message requests hitting the servers.  The suggestion has been to go ahead and try, which is easier said than done inasmuch as increasing the default is easy – rolling it back perhaps not so (given it would require a further update, which people might opt to ignore unless part of a larger release – which then might not appear for several months after the original change).
  • In terms of cache, that is also a problem, as the limits do tend to be low (based on older computer specs), plus the viewer tends to have multiple and distinct caching structures, none of which are particularly efficient – although the Lab is trying to address some of them (hence the Simplified Cache RC viewer).
• There is a very short discussion on Voice options, given Vivox will be ending the Vivox 4 API. As this in more a viewer question, no conclusions could be drawn.

The following notes were taken from the Tuesday, October 12th, 2021 Simulator User Group (SUG) meeting. The meeting was recorded by Pantera Północy, and the video is embedded at the end of this summary. Note this summary focuses on the key points of the meeting; where there is something to report, the video should be referred to should full details of the meeting wish to be reviewed.

Server Deployments

The planned deployment for this week has been postponed due to late-breaking issues, and so will not be deployed for a week. However, regions that have not been restarted in 10 or more days will be restarted. See the (“lack of”) deploy plans notes for more.

SL Viewer

There have been no updates to the current crop on official viewers to mark the start of the week, leaving the pipelines as:

• Release viewer: version version 6.4.22.561752, formerly the CEF Update RC viewer, issued July 24 and promoted August 10.
• Release channel cohorts:
  • Apple Notarisation Fix RC viewer, version 6.4.23.564172, issued September 24 – this should remove the warning messages which are currently popping up.
  • Maintenance RC viewer updated to version 6.4.23.564063, on September 21.
  • Simplified Cache RC viewer, version 6.4.23.562623, dated September 17, issued September 20.
• Project viewers:
  • 360 Snapshot project viewer, version 6.4.23.563579, issued September 3.
  • Performance Floater project viewer, version 6.4.23.562625, issued September 2.
  • Mesh Optimizer project viewer, version 6.4.23.562614, issued September 1.
  • Legacy Profiles viewer, version 6.4.11.550519, dated October 26, 2020.
  • Copy / Paste viewer, version 6.3.5.533365, dated December 9, 2019.

Region Crossings

• The subject of region crossings came up again, specifically in reference to multiple sequential crossings via vehicle, and the problems that can occur (passing from one region to the next, and then on to the next before all relevant data between the first two has been fully transferred). There should be code in place to handle this, but the Lab acknowledges there may be cases there this doesn’t work well.
• Rider Linden acknowledged that more work is required on the entire physical region crossing protocol, but that, “it may involve starting from scratch and rethinking how the entire protocol works. That’s going to be a big job.”
  • The question here is, how best to delay interpolation to ensure all information on a vehicle and its passengers is received by one region so all of it can be passed on to the next. User Animats submitted a code contribution to Firestorm in 2018 (which has since been further revised) to help with this, but it is not perfect. The problem here is, too much delay  – say more than around 1/2 a second – is noticeable and can impact immersion; a second problem is, what may help ease some types of region crossing may make others more noticeable.
• Another problem is that physical / vehicle region crossings are such that there is little opportunity for any kind of “pre-transfer” of vehicle and avatar data, until the vehicle is on top of / crossing the actual region boundary (the 1m boundary). This is because there is no guarantee that a vehicle will turn away from a crossing without actually moving between regions – so simulator time (on both sides) is taken up in handling the pre-transfer without no point to the exercise.

Another option (again, to stress **as an example**) would be to always have up to date data on all adjacent regions. But that would cost us real money. How do we recoup that increased cost in a way that is fair to the people who actually make use of the increased data availability? I’m just trying to give examples of why none of these solutions are “free” or “simple”.

– Mazidox Linden on region crossings + potential solutions

• The suggestion was made to run regions on virtual machines, such that adjacent regions are on the “same machine”, removing the need for transferring data between different physical simhosts. The problems here are:
  • a) The number of “adjacent” regions can be huge (e.g. Blake Sea and the surrounding private estates + Mainland continents, and the Mainland continents as a whole).
  • Even if broken down into more manageable groups of regions all on the same hardware, but this again doesn’t entirely eliminate problems, an will result in some region crossings appearing smoother, and others (where they remain between different hardware) appearing “worse” by comparison.
  • Also, if the hardware fails fails running a batch of virtual machines, that’s potentially a larger number of regions that go with it than is currently the case. And while hot shadowing is possible so that if a server does fail, it’s shadow can automatically take over, that’s doubling overall hardware requirements + associated costs, which would have to be met somehow.
• As it is, the move to AWS has seen an overall improvement in region crossings, primarily because the hardware and infrastructure available via AWS is a lost more recent, and so more powerful (hardware) and faster (network) than the Lab’s old infrastructure.
• Whilst not just related to region crossings, an experiment on the Lab’s to-do list is to try to group clusters of regions by hardware, something that has not been tried in some time.

In Brief

• A major focus on the server  / simulator side of Second Life remains the work in updating the tools the Lab has at its disposal, which is to be followed by / overlap with a major operating system upgrade (not to 64-bit, which is viewed as a “humongous” piece of work, but one that will eventually need to be addressed, depending on the platform’s continued longevity).
• There is a brief discussion at the end of the meeting concerning mesh decimation, avatar meshes, rendering, and possible improvement, much of which is a subject of CCUG meeting discussions.