2018 SL project updates 11/2: TPVD meeting with Ebbe Altberg

Realm Of Light; Inara Pey, February 2018, on Flickr Realm Of Lightblog post

The following notes are taken from the TPV Developer meeting held on Friday, March 16th 2018. A video of the TPVD meeting is embedded below, my thanks as always to North for recording and providing it. Time stamps in the text below will open the video in a new tab at the relevant point of discussion.

This meeting was somewhat extended  – lasting 1 hour 30 minutes – as a result of the presence of Ebbe Altberg, Linden Lab’s CEO, who commented on some of the item of discussion that came up at his session at VWBPE 2018 (see my notes and transcript here), as well as more broadly discussing Second Life and Sansar.

SL Viewers

[0:17-2:20] There have been no further updates to the current release, RC and project viewers in week #11, leaving the pipeline as follows:

  • Current Release version, dated February 23, promoted March 1 – formerly the Nalewka Maintenance RC – No change
  • Release channel cohorts :
  • Project viewers:
  • Linux Spur viewer, version, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

General Notes

  • The Media Update RC viewer is unlikely to be promoted to release status in the immediate future, as it has some Windows 7 update issues which need to be resolved.
  • The Love Me Render viewer is making good progress, although it also has a Windows 7 issue.
  • Despite the issues with it (see my update here), the 360-snapshot project viewer is not getting a lot of attention.
  • Animesh project viewer is getting close to a possible RC release and the Animesh project close to a move to the main grid.
  • The Bakes on Mesh viewer has cleared LL’s QA, so a Bakes on Mesh project viewer for use on Aditi should be appearing soon.

New Viewer Caps

[3:51-7:04] The lab is introducing two new viewer caps they’d like TPVs to adopt quickly:

  • One will be used when the viewer first logs-in to read all of the deferred IMs received while the user was off-line, which are being moved from UDP delivery to HTTP in an attempt to overcome issues of off-line IMs failing to show.
  • The second is to read the correct set of abuse report categories from the server, so only valid categories are displayed within the viewer, allowing users to more correctly file ARs, rather than using invalid categories held viewer-side.

Texture Upload Issues on 32-Bit Viewers

[7:30-8:30] There is a known issue of 32-bit version crashing when trying to upload texture larger than 1024×1024 (remembering up to 2048×2048 can be uploaded, although they are compressed to 1024×1024). This appears to be a KDU issue, and has been reported.

General Discussion with Ebbe

Highlights only – refer to the video for the full discussion.

Economic Model


  • The Lab is looking to try to pivot the SL economic model away from a heavy reliance on land fees, and then in time hopefully reduce the cost of land.
  • This will see a shift from land to revenue generation through fees in other areas.
  • The Lab cannot simply drop land fees and raise fees elsewhere, the two have to be balanced, so while the Lab is hoping to “aggressively” tackle pivoting revenue generation, they will also be cautious in making changes.
    • The hope of reducing land fees can be seen in the reduction in Mainland fees.
    • [10:50 via Oz and Linden] The Mainland price reduction has already seen a significant uptick in interest for abandoned Mainland, with support being “overwhelmed” with requests.
  • There is currently nothing planned for Private region fees, simply because the Lab has to be cautious around revenue.
    • It might be a case of (a) fee increase(s) elsewhere first, followed after a time by consideration of what can be done with Private region fees.
    • It is however, something the Lab would like to do.
    • The steps must be measured not only to safeguard LL’s revenue stream, but also so as not to upset the SL economy.
    • For this reason, the Lab will take a little time to measure the Mainland restructuring before they make other changes, so that they can more accurately measure cause and effect between different types of change.
  • These ideas were also discussed that the VWBPE session with Ebbe – see my transcript notes (with audio from that session) for more.


[15:39-18:36] A general overview of Sansar – which is still is  Creator Beta – including the drive this year to gain an audience for Sansar, plus improvements to the VR aspect of the platform. Most of this is covered in my weekly Sansar updates. For Ebbe in particular, the Sansar team is at a point where he feels comfortable pivoting attention away from that platform and back to Second Life, including spending more time in-world.

Sansar and SL


  • A re-iteration that Sansar was never intended to be a replacement for Second Life.
  • Both products now have completely separate teams working on them
    • At VWBPE 2018, Ebbe indicated that the core SL team – engineering, development, operations, support – is “close to” 100 in number.
  • There is an area of overlap between the two products, but there are also very clear differentiators.
  • Proof that Sansar isn’t a replacement for Sl is the level and speed with which LL has continued to invest in SL (overhauling the viewer and simulator build mechanisms, bringing more performance and stability to the platform) and to add new capabilities (Bento, Animesh, Bakes on Mesh, EEP, rendering enhancements, etc.).

Moving SL to the Cloud


  • Progress is being made.
  • Experimental regions have been run in the cloud, and they worked.
    • There are a lot of functional limitations that must be addressed before regions users can access can be run in the cloud.
    • The regions did achieve a reasonably high concurrency level (precise number not given).
  • Much of what SL does natively – dynamically spinning-up a new set  of inventory management servers or a new set of log-in services, etc. – is similar in nature to a lot of what cloud service providers do, so a lot of the back-end work involved in moving to the cloud is taking what the Lab have, and adapting it to run within the infrastructure of the cloud.
  • It is a massive engineering undertaking that will take time.
  • Once completed, it is hoped operating SL in the cloud will allow LL to offer benefits to users, which might potentially include:
    • Reduced costs for regions that are spun-down and stored to disk when no-one is using them, should this be explored for Second Life
    • Ability for simulators and services to be more geographically based (e.g. simulators used largely by an audience in South America could be hosted in facilities in South America)
    • Ability to potentially have a broader cross-section of land product based on server types, with a broader range of performance / pricing.
  • It is hoped that, for the most part, users won’t be aware of services being switched from the Lab’s dedicated infrastructure to running within a cloud infrastructure
    • Some non-user facing services are already running in the cloud.
    • The work will be done progressively, and not a “flipping of the switch” for “everything”.
    • There is not end-date for the work. The Lab is approaching it as aggressively as possible, but there are a lot of technical hurdles to be cleared along the way, some of which will only become apparent as attempt are made to shift things and put them into production via the cloud.
    • To deal with potential issues / hurdles, it is possible that further ahead, there is a simulator RC channel “in the cloud”, while others are still running on the Lab’s own infrastructure.
  • Also see Ebbe’s comments from VWBPE 2018.

Upcoming New User Experience

[1:01:23-1:07:11] The Product Team is doing a lot of work with the new user flow, and are getting close to where they can start experimenting with ideas.

Part of this work involves a themed learning island reached via a new user clicking a themed ad which takes them to a themed landing page on the SL website, where they can sign-up, obtain an avatar in keeping with the theme, and are delivered to a learning island that also follows the theme.

This approach will be tested alongside the current on-boarding routes.

Interesting tidbits:

  • The lab spent over a year building a “fairly sophisticated” tracking system to gather data on new users and see how they are doing, i order to try to learn more about on-boarding / retention.
  • The Lab’s data / testing suggests new user retention is no better in welcome areas with greeters, than for those without greeters.
  • A test the Lab carried out using a (non-public) browser-based means of accessing Second Life from sign-up (no need to download the viewer) also did not achieve better retention than the “traditional” sign-up and download route.

Other Items

  • [32:10-34:07] Mirrors in SL: the inevitable discussion – and no, mirrors aren’t in Sansar!
  • [34:08-37:08] SL and VR: re-cap of why the VR viewer was dropped from Second Life – unable to maintain the comfort-level VR frame rates (90 fps). Also segues into a discussion of the Sansar / SL Edit mode differences (also see Ebbe’s comments from VWBPE 2018).
    • [37:10-37:40] Sansar benefits to SL: Oz confirms that some of the rendering work with the atmospheric shaders to improve SL’s appearance is leveraging lessons learned with Sansar.
  • [38:35-43:50] Texture caching: the project to improve the viewer’s texture caching is still very active, and once completed, the Lab plain to look at other aspects of how the viewer caches data.
  • [44:17-47:09] Linux Viewer: no real change from my last update.
    • TPV have the same problem as LL re: Linux developers.
    • LL would like to see more from the Linux community get involved.
    • Suggestion is for Linux users to try running the Windows viewer under Wine.
  • [47:10-49:25] OpenGL and GPU/CPU divide: discussion on updating SL’s OpenGL version, which is already under consideration at the Lab. Broadens into a discussion of modifying SL’s rendering capabilities to make more use of more GPU shader capabilities for calculation (rather than being reliant on the CPU), and the risks (to users) this entails (as many SL users don’t use more modern hardware with GPUs capable of taking the load).
  • [51:58-59:58] Community Gateways: Discussion on the Community Gateway programme and attracting users. Includes mention of partners, Place Pages, etc. Ideas raised seen as something that could be put to the SL Marketing team under Brett Linden.
  • [1:12:52-1:14:14] Feature Requests: when filing a Feature Request JIRA, it is better to keep the request focused on a single idea which can be easily digested. Multiple ideas should be submitted via separate JIRAs so that meaning don’t become confused / the JIRA becomes to complicated to understand, etc. Multiple JIRAs around related ideas can also be related via identifier.

2018 SL project updates 11/1: Simulator User Group

Kamigami, Pandora Resort Town; Inara Pey, February 2018, on Flickr Kamigami, Pandora Resort blog post

Server Deployments

As usual, please refer to the server deployment thread for the latest updates.

  • There was no deployment to the Main (SLS) channel on Tuesday, March 13th, leaving it on server version 18# According to the deployment notes, the channel should have been restarted.
  • On Wednesday, March 14th, the main RC channels should be updated as follows:
    • TBC: no deployment to BlueSteel, leaving it on server version 18#
    • LeTigre and Magnum should both receive a new server maintenance package, 18#, comprising internal fixes and further simulator logging improvements.

SL Viewer


  • The Media Update RC viewer updated to version, on Monday, March 12th.
  • A new Maintenance RC viewer was release on Tuesday, March 13th. Version, code-named Ouzo, after the anise-flavoured aperitif. This viewer includes over 50 bug fixes and improvements.

Remaining Pipeline

Besides these two updates the official viewer pipeline remains as follows:

  • Current Release version, dated February 23rd, promoted March 1st – formerly the Nalewka Maintenance RC.
  • Release channel cohorts:
    • Love Me Render RC viewer, version, dated March 2nd.
  • Project viewers:
  • Linux Spur viewer, version, dated November 17th, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version, May 8th, 2015 – provided for users on Windows XP and OS X versions below 10.7.

Other Items

Meeting Reminders

  • There is NO Content Creation User Group meeting this week – the next CCUG meeting will be on Thursday, March 22nd at 13:00 SLT.
  • There is no Web User Group meeting this week – the next WUG meeting will be on  Wednesday, March 28th, at 14:00 SLT.

2018 SL project updates 10/2: CCUG summary

Queen of Dragons? Surrounded by Animesh dragons by Wanders Nowhere and used by Lucia Nightfire as Animesh test models

The following notes are primarily taken from the Content Creation User Group (CCUG) meeting, held on  Thursday, March 8th, 2018 at 13:00 SLT. For the purposes of Animesh testing, the meetings have relocated to the Animesh4 region on Aditi, the beta grid – look for the seating area towards the middle of the region. The meeting is chaired by Vir Linden, and agenda notes, etc, are usually available on the Content Creation User Group wiki page.

There is no video to accompany this update, notes are taken from my own audio recording of the meeting.


Project Summary

The goal of this project is to provide a means of animating rigged mesh objects using the avatar skeleton, in whole or in part, to provide things like independently moveable pets / creatures, and animated scenery features via scripted animation. It involves both viewer and server-side changes.

In short, an Animesh object:

  • Can be any object (generally rigged / skinned mesh) which and contains the necessary animations and controlling scripts in its own inventory  (Contents tab of the Build floater) required for it to animate itself.
  • Can be a single mesh object or a linkset of objects (link them first, then set them to Animated Mesh via the Build floater > Features).
  • Has been flagged as and Animesh object in the project viewer, and so has an avatar skeleton associated with it.
  • Can use many existing animations.
  • Will not support its own attachments in the initial release.
  • Will not initially include the notion of a body shape (see below).


Viewer Update

An updated to the Animesh project viewer occurred on Wednesday, March 7th with the release of version The focus of this update has been bug and crash fixes (e.g. correcting some of the issues of LODs getting stuck).

Current Progress

  • The major piece of work to be completed (other than further bug fixing) is performance profiling: looking at possible limits on tri count, LI, to ensure Animesh can scale across regions without becoming a major performance impact. This work takes object scaling into consideration.
  • Remaining bugs to be resolved include the animation timing issue, whereby an avatar entering a region were an Animesh object is already running may not receive the animations updates for the object. This is liable to require both viewer and server updates to fix.

Body Shapes

Providing a notion for Animesh objects to have a body shape is being considered as an initial follow-on for the project. If implemented, this would mean that Animesh creations could:

  • Make use of the shape sliders.
  • Utilise the Appearance / Baking Service.
  • Have a fully avatar-like inventory.
  • Make use of the server-side locomotion graph for walking, etc.

The assumption is that if the Lab look to add this functionality, it will be handled in much the same was as is the case for avatars. However, this aspect of the work will be opened up for discussion and ideas once it has been officially adopted as a follow-on project, rather than having speculative (and premature) discussions about how body shapes might be applied / managed.

Bakes on Mesh

Project Summary

Extending the current avatar baking service to allow wearable textures (skins, tattoos, clothing) to be applied directly to mesh bodies as well as system avatars. This involves server-side changes, including updating the baking service to support 1024×1024 textures, and may in time lead to a reduction in the complexity of mesh avatar bodies and heads. The project is in two phases:

  • The current work to update the baking service to support 1024×1024 textures on avatar meshes.
  • An intended follow-on project to actually support baking textures onto avatar mesh surfaces (and potentially other mesh objects as well). This has yet to fully defined in terms of implementation and when it might be slotted into SL development time frames.

This work does not include normal or specular map support, as these are not part of the existing baking service.

It is important to note that this project is still in its preliminary stages. Any release of a project viewer (see below) doesn’t mark the end of the project, but rather the start of initial testing and an opportunity for creators to have input into the project.

Project Viewer

  • Currently with the Lab’s QA team.
  • Will support a basic means of applying system bakes to mesh faces, although their will be caveats as to how well this initially works (e.g. matching to mesh UV maps).
  • The majority of this work is viewer-side, allowing mesh faces to be flagged with a special texture ID which allows a specific part of an avatar bake to be applied to them.

Feature Request JIRA

It is recognised that there are numerous questions / concerns relating to Bakes on Mesh (see my week #8 update). Because of this, and the lack of a specifications document, user Elizabeth Jarvinen (polysail) has started putting together a Feature Request JIRA outlining some of the specifics which need to be considered with regards to the project. Some of the ideas offered might be considered as a part of the initial Bakes on Mesh project, so might be left for a follow-up project. For more information on this follow the links below:

Project Arctan

This is the code-name for the project to re-evaluate avatar and object rendering costs (avatar complexity and Land Impact). Again, just to be clear on this project:

  • It is only just starting.
  • The aim is to make avatar complexity and Land Impact values more reflective of the actual “cost” to render avatars and objects. This might lead to ARC / LI values changing.
    • However, there are currently no details of how LI / avatar complexity values may change or when, as the Lab is still gathering data at this point in time.
    • The Lab is aware of the potential for increases in LI values to cause disruption, and if this is the case, they will seek to minimise the impact as far as possible.
  • The work is not related to Animesh – which will see a “basic” cost (in terms of geometric complexity) applied to Animesh objects (see performance profiling, above). Animesh objects may be subject to further revision as a result of Project Arctan’s findings.
  • It is hoped that Graham Linden, who is leading the project, will attend future CCUG meetings to discuss Arctan as the project progresses.

Other Items

  • PBR Shaders: there have been requests for the Lab to implement physically based rendering (PBR) – using realistic shading/lighting models along with measured surface values to accurately represent real-world materials. Such requests have been noted by the Lab. While adding support for PBR shaders has not been ruled out, it is seen as a significant undertaking, and is thus seen more as a “someday, maybe” piece of work, rather than being a part of the current SL enhancement roadmap.
  • Terrain texture improvements: terrain textures in SL suffer from a low texture density, causing blurring, etc. Again, the Lab is aware of this, and considering making improvements to terrain texture as a possible future project.

Next Meeting

The next CCUG will be on Thursday, March 22nd, 2018, and 13:00 SLT.

2018 SL project updates 10/1: Simulator User Group

Meraki Islands; Inara Pey, February 2018, on Flickr Meraki Islandsblog post

Server Deployments

As usual, please refer to the server deployment thread for the latest updates.

  • There was no deployment / restart to the Main (SLS) channel on Tuesday, March 6th, leaving it on server maintenance package 18#
  • No deployments are scheduled to either the Magnum or LeTigre RCs on Wednesday, March 7th, leaving them on server maintenance package 18# However, given it is now two weeks since these channels were last restarted, they should undergo a restart this week.
  • The BlueSteel RC should receive a new server maintenance package on Wednesday, March 7th. 18# is said to comprise “internal fixes”.

SL Viewer

360-Snapshot Viewer

The 360-snapshot project viewer updated to, intended to offer improved image resolutions and clarity. However, the update includes some significant issues of its own, including:

  • A failure to render in-world objects / Linden Water in 360-snapshots if Object-Object Occlusion (CTRL-SHIFT-O) is active.
  • A failure to render Linden Water (and possibly in-world objects) when saving snapshots to disk when using the Custom drop-down and leaving Constrain Proportions unchecked.
  • Include HUDs option in the snapshot viewer is inverted: when unchecked, HUDs are displayed in snapshots (and might appear in 360-snaps), when checked, HUDs are not displayed.
  • UI placement issues when the UI is displayed in the snapshot floater preview window – these issues do not translate to captured images, which display the UI correctly.

JIRAs have been raised against these issues.

Other Viewers

Apart from the 360-snapshot project viewer update, all viewers currently in LL’s pipelines remain unchanged from the end of week #9:

  • Current Release version, dated February 23, promoted March 1 – formerly the Nalewka Maintenance RC – NEW
  • Release channel cohorts:
  • Project viewers:
  • Linux Spur viewer, version, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

Region Crossing Issues Investigation

As noted over the last few weeks, user Joe Magarac (animats) has been digging into the viewer code handling region crossings (See Firestorm JIRA FIRE-21915),SL Forum thread, and this Google document, plus my last update here for more).

He has now come up with s further approach, using  a scripted means to turn off physics and freeze a vehicle on hand-off to the next region and then wait for the avatar data to arrive, using llGetObjectDetails (avatar, [OBJECT_ROOT]) to check for the presence of avatars. On confirming they are present, the vehicle resumes. Overall, the delay is a few milliseconds, and seems to work. However, as Rider Linden notes, Linden Lab still needs to address the crossing logic, although where such work might fit in the current schedule is unclear.

Environment Enhancement Project (EEP)

Project Summary

A set of environmental enhancements, including:

  • The ability to define the environment (sky, sun, moon, clouds, water settings) at the parcel level.
  • New environment asset types (Sky, Water, Days – the latter comprising multiple Sky and Water) that can be stored in inventory and traded through the Marketplace / exchanged with others.
  • Experience-based environment functions
  • An extended day cycle (e.g a 24/7 cycle) and extended environmental parameters.

This work involves simulator and viewer changes, and includes some infrastructure updates.

Current Status

“I got my first inventory objects on the test grid yesterday,” Rider noted at the Simulator User Group meeting. “I’m cleaning up the inventory code today and will wire the old editors in so you can create assets. I expect to have a viewer that people can play with very, very soon.”

2018 SL project updates 9/2: TPV Developer Meeting

Cece's Secret; Inara Pey, January 2018, on Flickr Cece’s Secretblog post

The following notes are taken from the TPV Developer meeting held on Friday, March 2nd 2018. A video of the meeting is embedded below, my thanks as always to North for recording and providing it. Time stamps in the text below will open the video in a new tab at the relevant point of discussion. Not this was a foreshortened meeting (20 minutes), with some lengthy periods of silence.

SL Viewers


  • The Nalewka Maintenance RC viewer, version (dated February 23rd) was promoted to de facto release status on Thursday, March 1st, 2018.
  • The Love Me Render RC viewer updated to version on Friday, March 2nd, which incorporates KDU and other security improves and (I believe) brings it to parity with the new release viewer.

All other official viewers in the pipe remain as per the start of the week:

  • Release channel cohorts:
    • Media Update RC viewer, version, February 15.
  • Project viewers:
  • Linux Spur viewer, version, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

An update to the Media update RC is anticipated to bring it to parity with the new release viewer, and a new update to the 360-snapshot viewer is also anticipated, which hopefully addresses the fuzziness / resolution issues inherent in the current version (see my hands-on overview – link above – for more on this).

Upcoming Viewers

Maintenance Viewer

[3:00-3:09] A new Maintenance RC view should be appearing in week #10 (week commencing Monday, March 5th, 2018).

Bakes on Mesh Project Viewer

[3:10-4:30] A Bakes on Mesh project viewer, suitable for use with Bakes on Mesh test regions on Aditi is with th Lab’s QA team, and may be appearing in week #10.

Bakes on Mesh is a project to extend the current avatar baking service to allow wearable textures (skins, tattoos, clothing) to be applied directly to mesh bodies as well as system avatars. It does so by allowing the faces on a mesh body / head to be defined in terms that the Baking Service can understand, allowing system layers to be applied to those faces, using 1024×1024 textures (rather than the 512×512 currently used by the Baking Service).

It’s important to note that the project viewer (with simulator-side support) is only the first phase of this project (essentially providing the 1024×1024 texture support), and there will still be much more to be done after the project viewer has been made available. Please refer to my Bakes on Mesh updates, which form a part of my weekly Content Creation User Group updates, for more specific information on this project.

Animesh Update

[7:56-9:06 – with audio break-up] The current focus remains:

  • Viewer-side bug fixes.
  • Performance profiling for Animesh complexity and LI costs.

The next step is to deploy updated simulator code in support of the limits so it can be put to the test.

DDoS Attack

[10:38-11:00] Some have experienced audio stream issues since the widespread amplified,UDP-based Distributed Denial of Service (DDoS) attack against memcached servers which took place across the Internet on Monday, February 26th / Tuesday, February 27th (memcached is a distributed memory caching system and is used to speed up dynamic database-driven websites and Internet-facing services by caching data and objects in RAM). As audio streams is handled by separate services (e.g. Shoutcast). It is possible they may have been suffering after-effects of the attack.

[11:05-11:56] The Lab has little to add to the blog post published by April Linden on the situation, other than they continue to harden systems and services. Overall, Second Life can come under frequent DDoS attack, but it myriad of parts are generally robust enough to withstand such attacks without them reaching the point of impacting users.

Off-Line IM Delivery Failures on Logging-in

[5:10-7:00] A project has been initiated to try to resolve problems with the viewer failing to receive all off-line IMs when a use logs in.

The cause appears to be that when a user logs-in, the simulator floods the viewer with UDP messages for the off-line IMs either before the viewer is ready to handle them, or as such a rate, messages are dropped.

To correct this, the Lab is making an API change in the simulator “in the next couple of weeks” and will be introducing a new viewer cap. Together, these will allow the viewer to indicate when it is ready to receive and handle off-line IMs, and ensure that the simulator correctly packages the messages for delivery, rather than simply sending a flood of items to the viewer.

These updates will hopefully be made available in a manner which allows TPVs to merge them relatively easily, rather than forming one part of a huge number of changes to the viewer.

In Brief

  • [4:35-4:53] Viewer Texture Memory Use: the Lab still plans to look at the question of texture memory use within the viewer, although it is noted this is turning into a “bit of a minefield”.
  • [12:02-13:05] Dullahan Linux: there have been requests for the Lab to provide a Linux Dulllahan build of the viewer (see this repository). Callum Linden has confirmed he’ll try to put further work into this before moving on to a new project, to at least leave it in a state others with more knowledge of Linux might be able to use.
  • [15:35-15:48] Disabling UDP inventory messaging: this is still being targeted for summer 2018, but there is currently no confirmed date.

2018 SL project updates 9/1: SUG, and DDOS

Les Reves Perdus; Inara Pey, January 2018, on Flickr Les Reves Perdusblog post

Server Deployments

As usual, please refer to the server deployment thread for the latest updates.

On Tuesday, February 27th, the Main (SLS) channel was updated with server maintenance package 18#, previously deployed to the three RC channels. This update was to directly address an odd viewer crash situation some users have experienced. Speaking at the week #8 Simulator User Group meeting, and reproduced here for completeness, Simon Linden said of the issue:

The server is doing some better checking on update data it sends to the viewer. We saw a very odd situation a week or two ago where the region was sending odd data and viewers would crash immediately. It went away after we restarted the region, and we think it was some memory corruption … FWIW, the server was sending a value of zero for a prim-code … which is totally invalid … There were also some other invalid data (like a zero’ed UUID) so my theory was memory corruption.

We didn’t have any other smoking guns. That region was fine after restarting, or when we tried our own copy. It was one of those mystery bugs, which we sometimes get since SL is so big and complex. We don’t know why it got that way, or how to make it happen again. we ended up making both the region and the viewer more robust. The underlying problem is still there and, assuming it happens again, will still cause problems.

(See also: BUG-214564.)

On Wednesday, February 28th, 2018 the three main RC channels should be updated as follows:

  • LeTigre and Magnum; no deployment, no restart, leaving them on server maintenance package 18#
  • BlueSteel should received a new server maintenance package, 18#, containing further simulator logging improvements and internal fixes. This is dependent upon what happens with the ongoing global DDoS attack over the course of the next 12-18 hours.

SL  Viewer

There have been no official viewer updates at the start of the week, leaving the various pipelines as per the end of week #8:

  • Current Release version, dated January 26, promoted February 7 – formerly the Voice Maintenance RC.
  • Release channel cohorts:
    • Nalewka Maintenance viewer, version, February 23.
    • Love Me Render RC viewer, version, February 21.
    • Media Update RC viewer, version, February 15.
  • Project viewers:
  • Linux Spur viewer, version, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

Log-in Issues

On Monday, February 26th and continuing through Tuesday, February 27th, large numbers of SL users experienced significant issues in trying to log-in as a result of a widespread UDP-based Distributed Denial of Service (DDoS) attack on the Internet as a whole (rather than specific to SL).

On Tuesday, February 27th, whilst still dealing with the situation, April Linden at the SL Ops team took time out to post on the situation in the official blog:

Hi everyone.

As I’m sure most of y’all have noticed, Second Life has had a rough 24 hours. We’re experiencing outages unlike any in recent history, and I wanted to take a moment and explain what’s going on.

The grid is currently undergoing a large DDoS (Distributed Denial of Service) attack. Second Life being hit with a DDoS attack is pretty routine. It happens quite a bit, and we’re good at handling it without a large number of Residents noticing. However, the current DDoS attacks are at a level that we rarely see, and are impacting the entire grid at once.

My team (the Second Life Operations Team) is working as hard as we can to mitigate these attacks. We’ve had people working round-the-clock since they started, and will continue to do so until they settle down. (I had a very late night, myself!)

Second Life is not the only Internet service that’s been targeted today. My sister and brother opsen at other companies across the country are fighting the same battle we are. It’s been a rough few days on much of the Internet.

We’re really sorry that access to Second Life has been so sporadic over the last day. Trying to combat these attacks has the full attention of my team, and we’re working as hard as we can on it. We’ll keep posting on the Second Life Status Blog as we have new updates.

See you inworld!
April Linden
Second Life Operations Team Lead

The attack led to a wide number of issues for SL users, from an inability to log-in to SL, through to disconnects or other problems. As Simon Linden explained in the Simulator User Group meeting on Tuesday, February 27th:

It’s not just logins … and often there is cascading problems. If one system gets attacked, it might actually be functioning fine but if the rack or network switch it’s on is overloaded, anything else connected there also has problems.

As it stands, the worst of the attacks appear to be over, at least where SL is concerned, but users should keep an eye on the official blogs for further updates, as well as grid status updates.