2018 SL project updates 16/1: Simulator User Group

La Virevolte; Inara Pey, March 2018, on FlickrLa Virevolteblog post

Server Deployments

As always, please refer to the server deployment thread for the latest updates.

  • On Tuesday, April 17th, 2018, the Main (SLS) channel was updated with server maintenance package 18#18.03.29.513939, previously deployed to the RC channels and containing internal fixes.
  • On Wednesday, April 18th, 2018, the major RC channels, BlueSteel, Magnum and LeTigre should all be updated with the same server maintenance package 18#18.04.09.514272, containing internal fixes and a fix for BUG-214702.

SL Viewer

With the exception Animesh project viewer (see below), there have been no updates to the current SL viewers thus far in week #16, leaving the pipelines as follows:

  • Current Release version 5.1.3.513644, dated March 27, promoted April 13 – formerly the media update RC – NEW
  • Release channel cohorts:
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

Animesh Project Viewer Update

The Animesh project viewer updated on Monday, April 16th. Version 5.1.4.514468 brings the viewer to parity with the current release viewer. In addition this viewer has revised streaming cost/land impact formula for Animesh objects, which are also reflected in ARC (avatar rendering cost) calculations for Animesh items.

In summary, the updates are:

  • Animesh attachment limit = 1: only one Animesh object can be attached to an avatar at a time. This is unchanged from the original estimates.
  • Triangle Count Limit = 100,000: an animesh object (linkset) can have at most 100k triangles, where the count is based on the estimated size of the largest LOD (normally this is the high LOD). This includes all mesh triangles, static or rigged.
  • Land Impact: streaming cost = 15.0 + 1.5 * ktris + cost of non rigged prims: for a rigged mesh prim in an animesh linkset, the streaming cost will be 0.0015 * effective_tri_count – that is, 1.5 per thousand triangles. The value for effective_tri_count is derived from the estimated triangle count of the various LODs in the prim as follows:
    • High LOD: all of the estimated triangle count included in the effective_tri_count.
    • Medium LOD, Low LOD and Lowest LOD: the allowed number of triangles can be up to ½ of the LOD above, or 64, whichever is larger (i.e. Medium can be up to ½ of High, or 64, whichever is larger). If there are more triangles than this limit, that excess will be added to the effective_tri_count.

See Vir’s explanation in the Animesh updated limits and cost formulas forum thread for a complete explanation of these limits and how they have been arrived at.

An important point to note is that these formulas only apply to Animesh; there is a second, and longer-term project – ARCTan – a re-evaluation of all object and avatar rendering costs (and which may see further changes to Animesh calaculations). It is hoped that overall, ARCTan  will improve viewer-side performance and provide creators with positive incentives to build more performant content.

You can find out more on ARTan in this blog post and this blog post in this blog.

Viewer Texture Cache

As noted in several of my TPV Developer meeting updates, Linden Lab are trying to improve viewer caching – starting with the texture cache. Commenting on the work, Oz Linden said, “We’re experimenting with a number of different changes. Some that you might think (I did) would make things better turned out not to, but we’re making progress.” It’s not clear if / when any project viewer utilising any new texture caching capability will be available for general use.

LlRequestUserKey and LlNameToKey

The Lab has released two new LSL functions: llRequestUserKey and llNameToKey, both of which are in connection to the upcoming return of Last Names (see this blog post and this blog post for more):

  • llRequestUserKey:
    • Requests the Agent ID for the agent identified by name from the dataserver. The name given may be either the current name of an avatar or a historical name that has been used in the past. If no agent can be found with the supplied name this function returns the value NULL_KEY.
    • It returns a handle (a key) that can be used to identify the request when the dataserver event is raised.
    • Note that agent being searched for with this function does not need to be signed on to Second Life.
    • See the llRequestRequestUserKey wiki page for more.
  • llNameToKey:
    • Returns a key the Agent ID for the named agent in the region. If there is no agent with the specified name currently signed onto the region, this function returns the value NULL_KEY. Names are always provided in the form “First[ Last]” or “first[.last]” (first name with an optional last name.)
    • If the last name is omitted a last name of “Resident” is assumed. Case is not considered when resolving agent names.
    • Uses a different mechanism to look up agent information to the older llKey2Name().
    • See the llNameToKey wiki page for more.
Advertisements

2018 SL project updates 15/1: Simulator User Group

Spirit of Sun; Inara Pey, March 2018, on FlickrSpirit of Sunblog post

Server Deployments

As always, please refer to the server deployment thread for the latest updates.

  • The Main (SLS) channel was not updated on Tuesday, April 9th, nor was there a restart.
  • On Wednesday, April 11th:
    • The BlueSteel and LeTigre will not be updated, and remain on server maintenance package 18#18.03.29.513939, containing internal fixes.
    • The Magnum channel will apparently receive a further revision to 18#18.03.29.513939.

SL Viewer

  • The Love Me Render RC viewer updated to version 5.1.3.514115, dated April 5th.
  • The Ouzo Maintenance RC viewer updated to version 5.1.3.514128, dated April 5th.

The remainder of the SL viewer pipeline remains unchanged from the end of week #14:

  • Current Release version 5.1.2.512803, dated February 23, promoted March 1 – formerly the Nalewka Maintenance RC – No change
  • Release channel cohorts:
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

Bakes on Mesh

This is the project to extend the current avatar baking service to allow wearable textures (skins, tattoos, clothing) to be applied directly to mesh bodies as well as system avatars. This involves server-side changes, including updating the baking service to support 1024×1024 textures, and may in time lead to a reduction in the complexity of mesh avatar bodies and heads.

This work does not include normal or specular map support, as these are not part of the existing baking service.

As noted in my last CCUG update, the project viewer, version5.1.3.513936, arrived on March 30th, 2018 (see the Bakes on Mesh JIRA filter for reported issues). There is a forum thread on the project, which carries a degree of misinformation on the project. Siddean Munro has sought to clarify things in a blog post, and Cathy Foil has produced an introductory video as well.

Environmental Enhancement Project

This is the project to create a set of environmental enhancements including the ability to define the environment (sky, sun, moon, clouds, water settings) at the parcel level; new environment asset types (Sky, Water, Days – the latter comprising multiple Sky and Water) that can be stored in inventory and traded through the Marketplace / exchanged with others; Experience-based environment functions and an extended day cycle (e.g a 24/7 cycle) and extended environmental parameters. The work involves simulator and viewer changes, and includes some infrastructure updates.

Rider Linden is working on EEP, and he reported, “I’ve been dealing with a little bit of a tangent having to do with internal inventory. Turned out to be a bit more complex than I anticipated when I said: “Sure, I’ll just knock that off as part of EEP”. It is almost resolved and then I can get back to direct EEP development.”

He also revealed more on how EEP will be “layered”: “EEP will have four different altitude layers. (Right now I’m operating with ground, 1km, 2km and 3km.) … The use case I see for the layers isn’t so much skyboxes, more for landing areas for RP sims that may want their own atmosphere; orbiting space stations , etc. (and the layers need to be set by the parcel owner.)”

Aditi Inventory Sync

Usually, sync on inventory from Agni, the Main grid, to Aditi, the Beta grid is designed to take place during a routine syncing operation run at around 2:00am PST daily, triggered by an Aidit log-in (see here for more).

However, as most of those who routinely log-in to Aditi will know that the system isn’t working correctly, and often, a ticket is required to get your Aditi inventory correctly synchronised with your Agni inventory. Commenting on the situation at the Simulator User Group, Mazidox Linden said, “We’ve got a dev working on the inventory sync to Aditi. We’re making some interesting progress, hopefully more news soon.” Oz Linden further indicated that a fix is in the pipeline.

 

2018 SL project updates 12/1: Simulator User Group

Sanctuary; Inara Pey, February 2018, on FlickrSanctuaryblog post

Server Deployments

As always, please refer to the server deployment thread for the latest updates.

  • There was no deployment / restart to the Main (SLS) channel on Tuesday, March 6th, leaving it on server maintenance package 18#18.02.12.512536.
  • The main RC channel should be updated as follows on Wednesday, March 21st:
    • The BlueSteel RC should receive a new server maintenance package. 18#18.03.17.513365 is said to comprise “internal fixes”.
    • The LeTigre and Magnum regions should receive a new server maintenance package. 18#18.03.14.513292, comprising internal fixes, additional logging to help diagnose an issue with in-world HTTP servers returning HTTP 503 (BUG-214702); add two new capabilities: one to request IMs that were delivered to the requesting agent while the agent was off-line; one to request the most up to date list of abuse report categories.

New Capabilities

The new capabilities in 18#18.03.14.513292 for Magnum and LeTigre are the first part of a set of server an viewer updates.

  • The new IM cap is to overcome of off-line IMs failing to be delivered when a user logs in. Currently, these are delivered via UDP, whether or not the viewer is ready to receive them. With the new capability (once grid-wide and implemented within the viewer), the viewer will request off-line IMs, which the server will package and deliver to the viewer via HTTP.
  • The new abuse report cap will replace the need for the viewer to have AR categories hard-coded into it. Once fully deployed, and a viewer update released, it will mean the view will request the current list of AR categories from the server when starting up, making the management of the list easier, and hopefully reducing the number of ARs filed under outdated categories.

SL Viewer Updates

There have been no updates to the current selection of official viewer at the start of the week, leaving the pipelines as follows:

  • Current Release version 5.1.2.512803, dated February 23rd, promoted March 1st – formerly the Nalewka Maintenance RC.
  • Release channel cohorts
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17th, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8th, 2015 – provided for users on Windows XP and OS X versions below 10.7.

Region Crossings

As noted over the last few weeks, user Joe Magarac (animats) has been digging into region crossings, particularly looking at the issue of avatars getting unseated from vehicles (See Firestorm JIRA FIRE-21915, this SL Forum thread, and this Google document, plus my last update here for more).

His latest approach involves adding a script to a vehicle effectively turns off physics and freezes the vehicle when received by a new region until it can confirm the associated avatar data has arrived (using llGetObjectDetails (avatar, [OBJECT_ROOT]) to check for the presence of avatars). On confirming the avatars are present, the script check the vehicle is allowed to resume.

“The typical freeze time is 40-120ms in addition to the regular region crossing delay, so you barely notice.” Joe told me in discussing the approach, which he has now made available to those wishing to test it using a simple motorbike build. “For double region crossings, the vehicle should slow down, although not by too much. I’ve also tried the same script in aircraft, and it seems to work.”

Joe continued, “Right now, I want to see if it ever fails. Issues are logged to an external server, so I can see what’s going on. I’d also love to get some of the SL motorcycle clubs involved in testing , so I’m giving out these test bikes. However, if anyone wants to test the script with their own vehicles, they can contact me for a copy, and I’ll be happy to work with them.”

Again, this workaround isn’t intended to solve all region crossing issues (e.g. it won’t stop “rubber banding” or address motion prediction – which the Lab sees as probably too aggressive), but it should help with full or partial avatar unsits; so those interested in given it a go are welcome to drop Joe a line.

Other Items

SL Messaging Layer

“I’m looking at the messaging layer; that seems to be the problem behind a lot of laggy things,” Simon Linden stated during the SUG meeting, revealing some of his recent work.

“There’s been a lot of learning. That code is complicated and there’s actually a lot of wisdom written into it, but I’ve found it doesn’t really do the best things when packet loss increases. The most basic thing I’ve found is that the estimated times based on pings used for time-outs can grow too fast: like on a 100ms connection, it can jump up to using 5-10 seconds. There’s a balancing act between resending data, and thinking it needs tons of time to get there.

“It really goes bad if a ping packet or the reply gets lost. SL just uses the same time-out for each retry – that’s another part that could be smarter.”

Defining this as an “interesting” part of the code, he’s reasonably certain it could be doing things better – and in doing so, could also help with region crossings and overall updates.

He also noted that also the SL bandwidth code is broken. This appears largely due to the volume of traffic going over the HTTP caps and via the CDN(s), which isn’t counted. “So I’m pretty sure we clog up a narrow pipeline more than we should,” he added.

2018 SL project updates 11/1: Simulator User Group

Kamigami, Pandora Resort Town; Inara Pey, February 2018, on Flickr Kamigami, Pandora Resort blog post

Server Deployments

As usual, please refer to the server deployment thread for the latest updates.

  • There was no deployment to the Main (SLS) channel on Tuesday, March 13th, leaving it on server version 18#18.02.12.512536. According to the deployment notes, the channel should have been restarted.
  • On Wednesday, March 14th, the main RC channels should be updated as follows:
    • TBC: no deployment to BlueSteel, leaving it on server version 18#18.03.05.513046.
    • LeTigre and Magnum should both receive a new server maintenance package, 18#18.03.12.513258, comprising internal fixes and further simulator logging improvements.

SL Viewer

Updates

  • The Media Update RC viewer updated to version 5.1.3.513038, on Monday, March 12th.
  • A new Maintenance RC viewer was release on Tuesday, March 13th. Version 5.1.3.513234, code-named Ouzo, after the anise-flavoured aperitif. This viewer includes over 50 bug fixes and improvements.

Remaining Pipeline

Besides these two updates the official viewer pipeline remains as follows:

  • Current Release version 5.1.2.512803, dated February 23rd, promoted March 1st – formerly the Nalewka Maintenance RC.
  • Release channel cohorts:
    • Love Me Render RC viewer, version 5.1.3.513005, dated March 2nd.
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17th, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8th, 2015 – provided for users on Windows XP and OS X versions below 10.7.

Other Items

Meeting Reminders

  • There is NO Content Creation User Group meeting this week – the next CCUG meeting will be on Thursday, March 22nd at 13:00 SLT.
  • There is no Web User Group meeting this week – the next WUG meeting will be on  Wednesday, March 28th, at 14:00 SLT.

2018 SL project updates 10/1: Simulator User Group

Meraki Islands; Inara Pey, February 2018, on Flickr Meraki Islandsblog post

Server Deployments

As usual, please refer to the server deployment thread for the latest updates.

  • There was no deployment / restart to the Main (SLS) channel on Tuesday, March 6th, leaving it on server maintenance package 18#18.02.12.512536.
  • No deployments are scheduled to either the Magnum or LeTigre RCs on Wednesday, March 7th, leaving them on server maintenance package 18#18.02.12.512536. However, given it is now two weeks since these channels were last restarted, they should undergo a restart this week.
  • The BlueSteel RC should receive a new server maintenance package on Wednesday, March 7th. 18#18.03.05.513046 is said to comprise “internal fixes”.

SL Viewer

360-Snapshot Viewer 5.1.3.513006

The 360-snapshot project viewer updated to 5.1.3.513006, intended to offer improved image resolutions and clarity. However, the update includes some significant issues of its own, including:

  • A failure to render in-world objects / Linden Water in 360-snapshots if Object-Object Occlusion (CTRL-SHIFT-O) is active.
  • A failure to render Linden Water (and possibly in-world objects) when saving snapshots to disk when using the Custom drop-down and leaving Constrain Proportions unchecked.
  • Include HUDs option in the snapshot viewer is inverted: when unchecked, HUDs are displayed in snapshots (and might appear in 360-snaps), when checked, HUDs are not displayed.
  • UI placement issues when the UI is displayed in the snapshot floater preview window – these issues do not translate to captured images, which display the UI correctly.

JIRAs have been raised against these issues.

Other Viewers

Apart from the 360-snapshot project viewer update, all viewers currently in LL’s pipelines remain unchanged from the end of week #9:

  • Current Release version 5.1.2.512803, dated February 23, promoted March 1 – formerly the Nalewka Maintenance RC – NEW
  • Release channel cohorts:
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

Region Crossing Issues Investigation

As noted over the last few weeks, user Joe Magarac (animats) has been digging into the viewer code handling region crossings (See Firestorm JIRA FIRE-21915),SL Forum thread, and this Google document, plus my last update here for more).

He has now come up with s further approach, using  a scripted means to turn off physics and freeze a vehicle on hand-off to the next region and then wait for the avatar data to arrive, using llGetObjectDetails (avatar, [OBJECT_ROOT]) to check for the presence of avatars. On confirming they are present, the vehicle resumes. Overall, the delay is a few milliseconds, and seems to work. However, as Rider Linden notes, Linden Lab still needs to address the crossing logic, although where such work might fit in the current schedule is unclear.

Environment Enhancement Project (EEP)

Project Summary

A set of environmental enhancements, including:

  • The ability to define the environment (sky, sun, moon, clouds, water settings) at the parcel level.
  • New environment asset types (Sky, Water, Days – the latter comprising multiple Sky and Water) that can be stored in inventory and traded through the Marketplace / exchanged with others.
  • Experience-based environment functions
  • An extended day cycle (e.g a 24/7 cycle) and extended environmental parameters.

This work involves simulator and viewer changes, and includes some infrastructure updates.

Current Status

“I got my first inventory objects on the test grid yesterday,” Rider noted at the Simulator User Group meeting. “I’m cleaning up the inventory code today and will wire the old editors in so you can create assets. I expect to have a viewer that people can play with very, very soon.”

2018 SL project updates 9/1: SUG, and DDOS

Les Reves Perdus; Inara Pey, January 2018, on Flickr Les Reves Perdusblog post

Server Deployments

As usual, please refer to the server deployment thread for the latest updates.

On Tuesday, February 27th, the Main (SLS) channel was updated with server maintenance package 18#18.02.12.512536, previously deployed to the three RC channels. This update was to directly address an odd viewer crash situation some users have experienced. Speaking at the week #8 Simulator User Group meeting, and reproduced here for completeness, Simon Linden said of the issue:

The server is doing some better checking on update data it sends to the viewer. We saw a very odd situation a week or two ago where the region was sending odd data and viewers would crash immediately. It went away after we restarted the region, and we think it was some memory corruption … FWIW, the server was sending a value of zero for a prim-code … which is totally invalid … There were also some other invalid data (like a zero’ed UUID) so my theory was memory corruption.

We didn’t have any other smoking guns. That region was fine after restarting, or when we tried our own copy. It was one of those mystery bugs, which we sometimes get since SL is so big and complex. We don’t know why it got that way, or how to make it happen again. we ended up making both the region and the viewer more robust. The underlying problem is still there and, assuming it happens again, will still cause problems.

(See also: BUG-214564.)

On Wednesday, February 28th, 2018 the three main RC channels should be updated as follows:

  • LeTigre and Magnum; no deployment, no restart, leaving them on server maintenance package 18#18.02.12.512536.
  • BlueSteel should received a new server maintenance package, 18#18.02.23.512831, containing further simulator logging improvements and internal fixes. This is dependent upon what happens with the ongoing global DDoS attack over the course of the next 12-18 hours.

SL  Viewer

There have been no official viewer updates at the start of the week, leaving the various pipelines as per the end of week #8:

  • Current Release version  5.1.1.512121, dated January 26, promoted February 7 – formerly the Voice Maintenance RC.
  • Release channel cohorts:
    • Nalewka Maintenance viewer, version 5.1.2.512803, February 23.
    • Love Me Render RC viewer, version 5.1.2.512751, February 21.
    • Media Update RC viewer, version 5.1.2.512574, February 15.
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

Log-in Issues

On Monday, February 26th and continuing through Tuesday, February 27th, large numbers of SL users experienced significant issues in trying to log-in as a result of a widespread UDP-based Distributed Denial of Service (DDoS) attack on the Internet as a whole (rather than specific to SL).

On Tuesday, February 27th, whilst still dealing with the situation, April Linden at the SL Ops team took time out to post on the situation in the official blog:

Hi everyone.

As I’m sure most of y’all have noticed, Second Life has had a rough 24 hours. We’re experiencing outages unlike any in recent history, and I wanted to take a moment and explain what’s going on.

The grid is currently undergoing a large DDoS (Distributed Denial of Service) attack. Second Life being hit with a DDoS attack is pretty routine. It happens quite a bit, and we’re good at handling it without a large number of Residents noticing. However, the current DDoS attacks are at a level that we rarely see, and are impacting the entire grid at once.

My team (the Second Life Operations Team) is working as hard as we can to mitigate these attacks. We’ve had people working round-the-clock since they started, and will continue to do so until they settle down. (I had a very late night, myself!)

Second Life is not the only Internet service that’s been targeted today. My sister and brother opsen at other companies across the country are fighting the same battle we are. It’s been a rough few days on much of the Internet.

We’re really sorry that access to Second Life has been so sporadic over the last day. Trying to combat these attacks has the full attention of my team, and we’re working as hard as we can on it. We’ll keep posting on the Second Life Status Blog as we have new updates.

See you inworld!
April Linden
Second Life Operations Team Lead

The attack led to a wide number of issues for SL users, from an inability to log-in to SL, through to disconnects or other problems. As Simon Linden explained in the Simulator User Group meeting on Tuesday, February 27th:

It’s not just logins … and often there is cascading problems. If one system gets attacked, it might actually be functioning fine but if the rack or network switch it’s on is overloaded, anything else connected there also has problems.

As it stands, the worst of the attacks appear to be over, at least where SL is concerned, but users should keep an eye on the official blogs for further updates, as well as grid status updates.