2019 SL User Groups 21/1: Simulator User Group

Amrum; Inara Pey, April 2019, on FlickrAmrumblog post

Server Deployments

At the time of writing, a deployment thread had yet to be posted however:

  • On Tuesday, May 21st, the SLS (Main) channel on Tuesday, May 14th, leaving regions on that channel on server maintenance package 19.05.10.527185, containing internal fixes.
  • On Wednesday, May 22nd, the three RC channels should be updated with a new server maintenance package, the details of which were still TBA at the time of writing this article.

SL Viewer

  • The Alternate viewer wiki page has now been retired and replaced by a new Alternate Viewers page.
  • The de facto release viewer was updated to version 6.2.2.527338, dated May 20th, formerly the updated Teranino Maintenance RC.

The rest of the current SL viewer channels remain as:

  • Release channel cohorts:
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

In Brief

  • Teleport Fail Issues: following the last Server User Group meeting on Tuesday May 14th, there was a test of a teleport fix Simon Linden had been working on, and a further test was carried out following this week’s meeting, with Simon Linden commenting:

Last week’s test was really helpful – it showed that one problem seemed to be fixed, but exposed another one that I hope is fixed this week. If it goes well, that new teleport code may go out onto RC channels next week.

  • Teleport Attachment Issues: this is a separate issue to that being investigated by Simon Linden, and which is more related to . there’s no current update on the work being put into fixing this, but as Simon observed in the meeting:

As far as I know, the problems with attachments tend to be with crossings and not teleports … while much of the code is the same, there’s usually a fundamental difference in crossings because you’re already connected to that region before you go there, whereas with teleports you are not … which was part of the big disconnect issue we had before.

  • Script Processing Issues: as reported in my week #18 SUG summary, there have been numerous reports of script run time issues, with some reporting that problems only started occurring following the roll-back on April 18th, 2019. BUG-226851 outlines some of the problems. The Lab are looking into this, but at present there are no updates on the problem.

 

Advertisements

2019 SL User Groups 20/1: Simulator User Group

La Virevolte; Inara Pey, March 2019, on FlickrLa Virevolteblog post

Server Deployments

At the time of writing, a deployment thread had yet to be posted however:

  • There are no deployment to the SLS (Main) channel on Tuesday, May 14th, leaving regions on that channel on server maintenance package 19#19.04.25.526669, primarily intended to correct the simulator-side EEP regressions that resulted of the roll-back of Thursday, April 18th. This update also includes a number of other internal fixes.
  • On Wednesday, May 15th, the three RC channels should be updated with two new packages:
    • BlueSteel and LeTigre should be updated with server maintenance package 19.05.01.526856, but not details are available at the time of writing.
    • Magnum RC will also be updated, but again, no details are available at the time of writing.

SL Viewer

  • The de facto release viewer was rolled back from the Teranino Maintenance RC release viewer, to the EAM viewer, version 6.2.0.526190, originally promoted on April 17th, 2019.
  • The Teranino Maintenance RC was re-released as a Release Candidate cohort on Monday, May 13th as version 6.2.2.527198.
  • The Love Me Render viewer updated to version 6.2.2.527021 on Monday, May 13th, 2019.

The rest of the current SL viewer channels remain as:

  • Release channel cohorts:
    • EEP RC viewer version 6.2.0.526104, dated April 11th.
    • Bakes on Mesh RC viewer, version 6.1.1.525409, dated March 26th.
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17th, 2017 and promoted to release status 29th November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, dated May 8th, 2015 – provided for users on Windows XP and OS X versions below 10.7. This viewer will remain available for as long as reasonable, but will not be updated with new features or bug fixes.

Network Maintenance

The first tranch of network maintenance announced last week has, according to the Grid Status page on the web, now been completed. This work included a period of time over the night of May 13th / 14th (US / EU) when Second Life log-in were disabled.

Commenting on the work, Simon Linden said, “Last night’s work isn’t anything that will be repeated … hopefully ever.”

While the core of the work has apparently been completed, Oz Linden noted there could be some final testing overnight on May 14th/15th, stating, “They’ll be doing some more testing tonight, but we don’t expect it to be (very) long or disruptive.”

Teleport Fail Issues

I’ve identified 3 different ways TPs could fail and I _believe_ I fixed those on the beta grid. Your help is really appreciated because the last time “someone” tried to fix things didn’t go well.

Simon Linden, SUG Meeting, May14th.

While the recent server-side deployment may have seen teleport success rates have returned to the levels witnessed before the recent “troubles”, the Lab continues to dig into problems, as per Simon’s comment above. To this end, those attending the SUG meeting were invited to a further test session, this one on Aditi (the Beta grid), to see how well his fix works. Expect an update following the next SUG meeting.

2019 SL User Groups 19/1: Simulator User Group

A L T I T U D E; Inara Pey, March 2019, on FlickrA L T I T U D Eblog post

Server Deployments

As always, please refer to the server deployment thread for the latest news.

  • On Tuesday, May 7th, the SLS (Main) channel updated to server maintenance package 19#19.04.25.526669, primarily intended to correct the simulator-side EEP regressions that resulted of the roll-back of Thursday, April 18th. This update also includes a number of other internal fixes.
  • On Wednesday, May 8th, the three RC channels should be updated with server maintenance package #19.05.01.526856, comprising internal fixes.

SL Viewer

The Teranino Maintenance RC viewer updated on Friday May 3rd, to version 6.2.1.526845. This viewer was subsequently promoted to de facto release status on Tuesday, May 7th.

The remaining viewer pipelines remain unchanged:

  • Release channel cohorts:
    • EEP RC viewer version 6.2.0.526104, dated April 11th.
    • Bakes on Mesh RC viewer, version 6.1.1.525409, dated March 26th.
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17th, 2017 and promoted to release status 29th November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, dated May 8th, 2015 – provided for users on Windows XP and OS X versions below 10.7. This viewer will remain available for as long as reasonable, but will not be updated with new features or bug fixes.

Teleport Disconnects

Following the deployments of server maintenance package 19#19.04.25.526669, it appears that there has been a significant improvement in teleports, with Simon Linden noting that the “TP crash has been mitigated and success rates are back to what they were months ago.”

This doesn’t mean that the Lab is ceasing efforts on trying to improve teleports and region crossings; rather that the priority of the work over that last few weeks has been on easing the problems people have been experiencing the last month.

Script Processing Issues

As reported in my week #18 SUG summary, there have been numerous reports of script run time issues, with some reporting that problems only started occurring following the roll-back on April 18th, 2019. BUG-226851 outlines some of the problems.

There is no specific update on this, other than the Lab’s engineering team has been asked to put together ways for them to look at script run times and performance grid wide.

2019 SL User Groups 18/1: Simulator User Group

Toshi Farms; Inara Pey, March 2019, on FlickrToshi Farmsblog post

Server Deployments

As always, please refer to the server deployment thread for the latest news.

  • There was no deployment of the SLS (Main) channel on Tuesday, April 20th, leaving all regions on that channel on server update 19#19.04.22.526534 comprising performance improvements (possible fixes for teleport / region crossing issues) and additional internal logging.
  • On Wednesday, May 1st, all three main RC channels should be updated to server maintenance package 19#19.04.25.526669, primarily intended to correct the simulator-side EEP regressions that resulted of the roll-back of Thursday, April 18th. This update also includes a number of other internal fixes.

SL Viewer

There have been no viewer updates at the time of writing this update, leaving the viewer pipelines as follows:

  • Current Release version 6.2.0.526190, formerly the Estate Access Management RC viewer, dated April 12, promoted April 17 NEW. – see my EAM overview for more information
  • Release channel cohorts:
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7. This viewer will remain available for as long as reasonable, but will not be updated with new features or bug fixes.

My.secondlife.com

My second Life has been down since the weekend of April 27/28th, 2019. This has affected all access to the site, the upload of snapshots and in-world web-based profiles.

Linden Lab is aware of the issue, and working to resolve the outage.

Teleport Disconnects

The teleport disconnect issue is now being seen as a race condition,with Simon Linden commenting:

The TP issue looks like a race condition … we “fixed” 3 other bugs and made that worse.

To which Oz Linden added:

And we’ve already put in place a bunch of improvements to our RC monitors to help prevent a similar recurrence (with more coming soon) … Doubtless Murphy will find new ways to avoid our monitors from time to time, but one tries.

The lab now feel they have investigated the issue and gathered sufficient data around it to be able to perform a post-mortem on the situation to (hopefully) reduce the likelihood of such an occurrence in the future.

Script Processing Issues

This was raised at the April 26th TPVD meeting as well. There have been numerous reports of script run time issues, with some reporting that problems only started occurring following the roll-back on April 18th, 2019. BUG-226851 outlines some of the problems.

As per the TPVD meeting, the Lab are not aware of anything that may have changed to impact script run time (particularly on Full regions); but the problem has been somewhat exacerbated by the issue being bounced between the JIRA and support (note how the above report has been closed, referencing the matter back to support).

2019 SL User Groups 17/1: SUG – teleport disconnects update

Puddlechurch; Inara Pey, March 2019, on FlickrUmiblog post

Server Deployments

From the server deployment thread for the week:

We are working on the TP & sim crossing disconnect issue, and making several changes over this week.  These may be a little bit more disruptive than our usual grid roll process and we apologise in advance for the inconvenience of those.  We’ll do our best to keep this disruption to a minimum. We have a simulator update which we will roll to BlueSteel and LeTigre on Tuesday.  Depending on the results we will make additional plans for gridwide rolls as soon as practical. We thank you for your patience and fully realize both the urgency and the frustration this has been causing.

Thus far, there has been a deployment to BlueSteel and LeTigre – server update 19.04.22.526534. Check the deployment thread for further updates.

SL Viewer

There have been no viewer updates at the start of the week, leaving the viewer pipelines as follows:

  • Current Release version 6.2.0.526190, formerly the Estate Access Management RC viewer, dated April 12, promoted April 17 NEW. – see my EAM overview for more information
  • Release channel cohorts:
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7. This viewer will remain available for as long as reasonable, but will not be updated with new features or bug fixes.

Teleport Disconnects

Let’s see … for server news, many of us are focused on the teleport issues … I hate to promise anything, but we all know a lot more about TP internal problems than we used to 🙂 . We do understand they’re annoying and a frustrating problem – we definitely want to fix it and make it better.

– Simon Linden, Simulator User Group, April 23rd

From the above statement, it should be clear that the Lab is devoting a lot of time to teleport disconnects. There have been mixed reports on the outcome of the deployment made on Thursday, April 18th, 2019, with some indicating matters have improved, others reporting no real change. In particular, people visiting Fantasy Faire and travelling around the new Linden Homes continent appear to be faring a lot better than had been the case. Which is not to say the issue has in anyway been resolved – hence the continuing work.

While it had been indicated that the recent operating system update may have played a role in the problems, Simon again referenced the timing issue with region crossings, whilst also mentioning the potential for the simulator side of EEP possibly also playing a role in things.

One of the puzzles we’re trying to sort out is if somehow the environmental work caused the TP problems – the timing of the release is suspicious but the functionality _should_ be different …. The disconnects are a problem where the viewer and 2nd region don’t start talking as they should [so something is out of sync] or there’s a failure to communicate. I know, for example, the 2nd region is waiting for the viewer to connect and get a message … that never happens …

He continued:

It doesn’t seem to be closely associated with AV complexity … that said, the more complex your AV is, the more work it needs to change regions. It’s always been better to have fewer scripts and data for teleports and region crossings.

The problem is still trying to pin down actual potential causes, with disconnects remaining inconsistent in terms of reproduction., again as Simon noted:

It’s frustratingly inconsistent. It’s a lot easier to fix something that breaks all the time … to fix it, and to know when you fixed it. For example, before this meeting we ran a test that had probably 400 or so successful teleports, no disconnects … that’s good, but not proof of a fix.

Mazidox Linden, from the Lab’s QA team further qualified the Lab’s problem:

We need in the neighbourhood of 10000 teleports to have any kind of real statistical confidence, just as a reference point 🙂 .

EEP / Windlight Issues

Following the Thursday deployment, many region holders / designers noted significant differences in how their region windlights were being rendered. The didn’t get much in the way of discussion during the meeting, however, Rider Linden offered an apology for the situation, noting that the focus on the TP issue(s) resulted in some unexpected regressions. However, it’s not currently clear what might be done to deal with this issue.

2019 SL User Groups 16/1: SUG – teleport disconnects update

Salt Water; Inara Pey, March 2019, on FlickrSalt Waterblog post

Server Deployments

As always, please refer to the server deployment thread for updates and news.

  • On Tuesday, April 9th the SLS (Main) channel was updated to server maintenance package 19#19.04.09.526122, previously deployed to the main RCs in week #15, and comprising logging improvements related to teleporting.
  • RC deployments were still TBA/TBC at the time of writing; I’ll update when more information is available.

SL Viewer

At the time of writing, there had been no viewer updates to mark the start of the week, leaving the SL viewer pipelines as follows:

  • Current Release version 6.1.1.525446, formerly the Love Me Render RC viewer, dated March 26, promoted April 2 NEW.
  • Release channel cohorts (please see my notes on manually installing RC viewer versions if you wish to install any release candidate(s) yourself):
    • Estate Access Management (EAM) RC viewer version 6.2.0.526190, April 12.
    • EEP RC viewer version 6.2.0.526104, April 11.
    • Bakes on Mesh RC viewer, version 6.1.1.525409, March 26.
    • Teranino Maintenance RC viewer version 6.1.1.525401, March 20.
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7. This viewer will remain available for as long as reasonable, but will not be updated with new features or bug fixes.

It is anticipated the EAM release candidate viewer could be  promoted to de facto release status this week.

Teleport Disconnects and Region Crossing Issues

Following the Simulator User Group meeting on Tuesday, April 9th, the Lab asked those attending to engage in a stress test of teleports to help gather data on the ongoing disconnect issues. Commenting on the results at this week’s meeting, Simon Linden also reiterated what the Lab believe is happening:

We have some good info on the underlying causes and are working on fixes, but don’t have a real improvement yet. it usually fails as the viewer and 2nd region don’t start communication properly, so I’ve been digging through that code and all the ways it can fail … that’s my life lately … I’m sure I’ll have something new for testing in a day or two so hopefully next week’s updates will be better, or at minimum give better information.

When asked if the issues with vehicle crossings are related to the same potential cause, he added:

 I haven’t looked specifically at vehicles so I can’t say, but it’s possible, yes. I tried a boat trip around a new Linden Homes continent and it was a challenge, to put it politely.

In the meantime, new code that might help relieve the issue was deployed to the Cake mini-RC channel on Tuesday, April 16th. Those wishing to test crossings on this channel can do so on the following regions: Morris, Arnthrud, Croak, Gouda, Impeller, Kasba, Oak Bay, Sapars, Yongma. Constructive feedback can be sent via IM or note card to Simon Linden.

When testing, please wait around 2-3 minutes between each teleport to allow you to fully disconnect from the region you have just departed, particularly if you intend to just bounce back and forth between the same two regions (data is retained for a short time by a region you’ve just left, in case you want to rapidly want to TP back, and this data needs to be dropped during testing, as it can affect results).