SL project updates week 36/2: TPV Dev meeting + SL in the cloud

Brand New Colony; Inara Pey, September 2017, on FlickrBrand New Colonyblog post

Updated to reflect the release of the Wolfpack RC, which appeared after this article had been uploaded for publishing.

The majority of the notes in this update are taken from the TPV Developer meeting held on Friday, September 8th 2017. The video of that meeting is embedded at the end of this update, my thanks as always to North for recording and providing it. Timestamps in the text below will open the video in a separate window at the relevant point for those wishing to listen to the discussions.

Server Deployments Week #36 – Recap

Please refer to the deployment notice for the week for latest updates and news.

Region Return Issues

Whether connected to the Wednesday deployment or the grid-wide issues experienced later that day, but some regions reported significant returns of in-world objects (Mother Road, on the Main (SLS) channel, which I blogged about here, suffered the problem, as Heavenly Views (Magnum RC) also apparently did). Most of these problems seem to have been rectified by a roll-back of the affected regions.

SL Viewer

A new RC viewer, codenamed Wolfpack – version – was released late on September 8th. This viewer is functionally identical to the release viewer, but includes additional back-end logging to “help catch some squirrelly issues”.

[1:15-4:15] Otherwise have been no further viewer updates since the start of the week, leaving the current pipeline as follows:

  • Current Release version, dated August 9, promoted August 23 – formerly the Maintenance RC
  • RC viewers:
    • Alex Ivy 64-bit RC viewer version, dated September 5th
    • Voice RC viewer updated version, dated September 1st
    • Maintenance RC viewer version, dated August 31st
  • Project viewers:
  • Obsolete platform viewer version, dated May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

The Alex Ivy and Voice updates bring these RCs into parity with the current release viewer. These updates have lowered the overall crash rates for both viewers, although the Voice viewer’s crash rate is still somewhat elevated compared to the release viewer. This elevated crash rate is something the Lab has seen in the various viewers for the last few months, and are working to lower.

Alex Ivy will likely have two more RC releases before it is ready for promotion to de facto release status, as there are a couple of issues the Lab wants to clear up before any promotion of this viewer.

It is hoped work will shortly be resuming on the 360-degree snapshot viewer, part of which will also be bringing it up to parity with more recent releases.

New Viewer Splash / Log-in Screen

[4:25-8:07] The Lab is in the process of revamping the official viewer’s splash / log-in screen. This should be inherited automatically by those TPVs using the Lab’s splash screen. This will see the screen have a different look and feel, with the information arranged a little differently, with the grid status made somewhat more prominent. Some of the information widgets associated with the splash screen are also being updated as a part of this work, and will require testing by TPVs when available (RSS feeds should not be affected).

This work is being carried out by Phronimos Linden, one of the more recent (3 months ago at the time of writing) Lab staff recruited to work on Second Life.

Moving to the Cloud

[15:52-25:25] As confirmed in a recent Lab blog post, Second Life services are moving to the cloud. There are no specific details on this as yet, but in short:

  • The work will be carried out in stages.
  • It will include providing regions from the cloud (rather than the Lab’s own co-located servers).
  • As many know, the asset services has been in S3 cloud servers for several years.
  • The order of moving services hasn’t been determined as yet, but the aim will be to move services, then test for reliability and performance before opening them up / moving to the next service(s).
  • Some testing has already begun, using services only accessed indirectly by users.
  • The Lab has been working on the infrastructure for this for a while, but none of the major services have been moved.

It is unlikely this will lead to regions of a different size to those presently on the grid, or which can be varied in size (a-la the OpenSim varregions), as region size is very deeply baked into the simulator and viewer code (and probably elsewhere in the overall SL code base); although some at the Lab would love to be able to have such a region capability. However – as indicated in the Lab’s blog post – it may allow the Lab to introduce new region products, possibly ones capable of handling greater loads.

It is hoped that this work will result in lower tier over time, but whether this will be the case of not is still and open question at this time, as it is unclear as to what costs will be involved in terms of cloud instance types., etc., that will be required to support simulators.

Overall, there is a lot the Lab hopes to achieve with the move, but the precise benefits are likely to only become clear once things have been tried and found to work / over time as the work progresses and beyond. However, at this point time scales are TBD, and it is liable to be some time before user-visible aspects of the service move to the cloud (although non-visible services – such as the log-in service, for example – could be moved sooner).

A precursor to this project is the continuing work to update the servers to newer version of Linux, work which is making “good progress”.

Other Items

Asset HTTP Messaging and Asset HTTP Issues

[10:11-10:45 and 12:10-15:23] With the promotion of the Asset HTTP viewer to release status in June 2017, the majority of assets are now delivered to the viewer using HTTP via the Content Delivery Network(s) used by the Lab. This means that UDP messaging for all of the affected asset types will be turned off at the server end at some point.  Currently no date has been set for this, and it looks like it may not occur much before February 2018.

There is still an issue related to fetching assets over HTTP in general which is experienced by some users some of the time   whereby something causes the pipeline textures to get out of sync and things to go awry. As the problem happens for some and not others, the Lab is still trying to determine the root cause for the problem, but it is a matter the Lab is trying to resolve.

Catznip viewer reports issues with textures over HTTP “stalling” for between 5-10 minutes on their pre-release viewer with the latest HTTP updates, but this isn’t something the Lab is aware of as a more general issue.

Estate Tool Ban List Improvements

[10:48-11:43] The Lab did some initial work to improve ban lists at the estate / region level a while ago, but the intended work to improvement to overall layout for the lists, etc., has been delayed do to work on dealing with viewer crashes, etc. This work is apparently now “next in line” once some of the existing viewer projects are shipped.

The conversation from 26 minutes to the end is more general chat on viewer stats, speculation on the reason for the texture load stalls, and an ad for the Firestorm birthday party.

One thought on “SL project updates week 36/2: TPV Dev meeting + SL in the cloud

  1. I think I have seen the HTTP stalling with the 64-bit Linux version of Firestorm but the Linux version of the SL Viewer refuses to run on my system, which seems to block all chances of help from anyone.

    I saw the Region problems on Thursday, but only on Magnum servers. If it was happening on the main channel as well, the total silence from the Lindens on Status and in the Server Tech forum looks pretty damning. It deserved at least the standard new-builds warning we get for rolling restarts.


Comments are closed.