Category: SL User Group Meetings

SL project updates week 10/2: server and viewer notes

Posted on by Inara Pey
RMS Titanic, Second Life
RMS Titanic, Second Life

Server Deployments

Tuesday, March 3rd, saw the Main (SLS) channel receive the server maintenance package deployed to the RC channels in week #9. This includes:

  • A server-side fix for BUG-8297, “Unable to teleport anywhere using SLGO”
  • Improvements to server logging.

There were no deployments to the RC channels on Wednesday, March 4th.

Upcoming Server Updates

During the Server Beta User Group meeting on Thursday, March 5th, Caleb Linden, who now chairs the meetings, indicated that an upcoming deployment to the RC channel will include a change to the number of off-line IMs delivered when you log-in.

Currently, the cap is set to 25, but it’s not clear what the new limit will be, although it might vary (or be adjustable) depending on whether a user is a Basic or Premium member. This change will not affect the forwarding of off-line IMs to e-mail.

SL Viewer

The new Tool Chain Update viewer was promoted to Release Candidate status on Wednesday, March 4th. Version 3.7.26.299443 see a change to the tools used to build the viewer (e.g. the use of Visual Studio 2013, Xcode 6.1, and some other tools improvements). This update also incorporates the log-in screen update to a single button, replacing the need for the Log-in Screen RC viewer.

Future Viewer Updates

Avatar Layers Global Limit

In 2014, BUG-6258, “Popularity of Mesh Attachments Facilitates Need For More Alpha Layers” is a request to raise the number of alpha layers which can be concurrently worn, prompted the Lab to consider the introduction of a global limit on the number of clothing layers an avatar could wear,so that, like attachments, an avatar can wear as many clothing layers in any combination, up to that limit, rather than having individual limits enforced per clothing layer.

During the TPV Developer meeting in week #9, Oz Linden indicated that while there had been major progress on this idea since it had been first raised, something would likely be appearing later in the year. Since then. Whirly Fizzle has spotted a viewer code repository from Vir Linden: “SL-103 WIP – viewer enforces wearable limits based on total articles of clothing rather than per-type limit. Limit is artificially low for testing, will bump before release.”

This doesn’t mean new global limits on avatar cloting layers will be appearing in the next few weeks, but it does indicate that it is one of several avatar related projects he’s working on (together with the new Avatar Hover Height feature and fixing at least some of the current crop of attachment issues).

Notifications Changes

Also during the TPVD meeting in week #9, Oz commented that some “cosmetic” work was being done to the way notifications are presented through the viewer. Again, thanks to Whirly’s poking around, this would appear to be in connection with BUG-8000, “Separate transaction notices from group notice/invites or provide some means for sorting incoming notifications”, which has been imported by the Lab and is being worked on.

A preliminary version of the proposed new notification panel (see BUG-8000). Note that is is a work-in-progress and may be subject to change prior to release (image courtesy of Whirly Fizzle)
A preliminary version of the proposed new notification panel (see BUG-8000). Note that is is a work-in-progress and may be subject to change prior to release (image courtesy of Whirly Fizzle)

 Group Chat

Simon Linden is continuing to bang on things to try to correct issues that have arisen since the last set of updates at the end of week #9. According to Caleb Linden, who now chairs the Server Beta Group meetings, those groups that are affected are seeing up to a 20% failure rate in delivered messages.

Other Items

Rigged Mesh Crashers

Further to my last report on this issue, Whirly Fizzle (who is genuinely one of the great powerhouses in helping to keep SL rolling along) stepped up to provide a more detailed explanation of the issue based on her own extensive investigations into the matter (thanks Whirly! 🙂 ).

In her comments, Whirly notes that Chalice Yao has developed a debug  setting within her own NACL viewer setting specifically stop cap rendering of worn rigged meshes, and that Nicky Dasmijn from the Firestorm team is working on a patch for their viewer. However, while the setting works, it can also prevent “legitimate” rigged mesh items with high vertices counts from rendering.

Further discussion of the problem with regards to BUG-2082, which includes new viewer rendering controls, confirmed this would not help with the issues presented by this particular kind of rigged mesh crasher but it will prevent issues such as the animated mesh “tail of doom” which can cause viewer crashes as a result of overloading memory.

SL project updates week 10/1: server, general news

Posted on by Inara Pey

Leka, Nordan om Jorden; Inara Pey, March 2015, on Flickr Leka, Nordan om Jorden (Flickr) – blog post

Server Deployments

Tuesday, March 3rd, saw the Main (SLS) channel receive the server maintenance package deployed to the RC channels in week #9. This includes:

  • A server-side fix for BUG-8297, “Unable to teleport anywhere using SLGO”
  • Improvements to server logging.

There were no scheduled deployments to the RC channels on Wednesday, March 4th.

Group Chat

Following the last deployment of back-end group chat changed during week #9, some large groups with active group chat have reported an increase in issues of message failures, although they appear to do so somewhat randomly, with some people seeing them and others simply not receiving them at all.

Commenting on the problem at the Simulator User Group meeting on Tuesday, March 3rd, Simon Linden summarised the situation thus:

In short, yes, it’s cranky, and yes, we’re (as in I am) looking at it … the chat server itself is actually running better than before, believe it or not. A back-end service it relies on, what we call “agent presence” [used to help locate someone on the grid], seems to be having new problems, so the changes may have added load to those servers and is causing problems, or something else unexpectedly changed … [So] some people don’t get the messages when chat is failing … it’s dropping sending some updates and messages when it times out with some other internal requests.

Further updates will be provided as the Lab / Simon continues to look at the problems.

CDN Notes

There have been recent reports of people experiencing slow texture and mesh load issues, leading to questions concerning the CDN service (although some of the issues that have been mentioned might be related to local caching more than the CDN). In particular questions have been asked as to how long a CDN server retained its cache of data relating to regions prior to going “cold” and requiring a “reload” from the SL services. Commenting on this at the Open-source Developer meeting on Monday, March 2nd, Oz Linden said that some CDN caches do age out more quickly than others.

The Lab has also been experimenting with more than one CDN provider, and are continuing with different CDN configurations as well to further tune things, as well as continuing to measure results; so we may yet see further changes  / improvements, and a possible decrease in instances that may be related to “cold” CDN loads.

Other Items

Rigged Mesh Crashers

The Server Beta Meeting on Thursday, February 26th saw the issue of a “new” mesh crasher being used on the grid. This is essentially a deliberately corrupted rigged mesh attachment which, when worn will cause viewers around it to immediately crash, with no warning or ability to take preventative action, such as muting the offending avatar.

Just over a year ago, some advice was given on how to counter graphics crashers by adjusting the viewer’s debug settings, and some people many be getting pointed towards it again in order to avoid being affected by the “new” crasher.

However, changing the specified debug settings can lead to a failure to render much of what you actually want to see, as noted in  this comment following the article. At the time the advice was given, the Firestorm team tracked many of the problems their users were experiencing directly to the settings having been changed. Ergo, if you are pointed to this particular article as a means of combating graphics crashers, please keep in mind you may gain undesirable results, and keep a note of the original settings so you can switch back to them should this be the case.

During the discussion on this matter at the SBUG meeting, speculation was raised on whether or not the forthcoming new viewer rendering controls (see: STORM-2082). opinion is divided, as the viewer downloads the data which may cause a graphic crash and starts processing some of it in order to determine what to render or not, and even this initial processing could be enough to crash it.

SL Feed Issues

There has been an uptick in the number of snapshot uploads to the SL feeds failing over the course of the last week, with some additionally reporting issues of comments failure to appear / “loves” failing to stick. Some users also reported issues over the weekend with web profiles failing to load, and a JIRA (see BUG-8677) was logged on this issue on March 3rd.

The last several days have seen people again encounter issues with snapshots failing to process / display in their feeds
The last several days have seen people again encounter issues with snapshots failing to process / display in their feeds

Whether the two issues had a common cause isn’t clear, but as the latter has been resolved, and you are one of those continuing to experience snapshot upload failures, please file a JIRA providing as much information as possible (links to any feed post with a missing snapshot, date / time of upload, number of failures, etc.).

SL project updates week 9/2: TPV meeting, SL roadmap notes

Posted on by Inara Pey

Less Than Three, Special One; Inara Pey, February 2015, on Flickr Less Than Three, Special One (Flickr) – blog post

The following notes are primarily taken from the TPV Developer meeting held on Friday, February 27th,  a video of which is included at the end of the article (my thanks as always to North for recording it and providing it for embedding), and any time stamps contained within the following text refer to it.

Server Deployments Week 9 Re-cap

As always, please refer to the server deployment thread for the latest news and updates.

  • There was no deployment to the Main (SLS) channel on Tuesday, February 24th
  • On Wednesday, February 25th, all three RC channels received the same server maintenance package, which comprises:
    • A server-side fix for BUG-8297, “Unable to teleport anywhere using SLGO”
    • Improvement to server logging.

SL Viewer

Viewer-Managed Marketplace Project Viewer

[0:35] As noted at the last TPV Developer meeting, the VMM functionality still has some issues that are being worked on. some of these appear to be on the back-end of things, rather than within the viewer itself. As such, it is unlikely that there will be further updates to the viewer in the near future while these are being addressed.

Experience Tools Viewer

[0:50] The Lab is also continuing to work on the back-end of things for Experience Tools / Keys. This work is apparently focused on overall robustness of the service prior to fully deploying it to the grid. As no viewer-side changes are anticipated as a result of this work, the viewer itself should be ready for promotion to the de facto release viewer once the Lab is ready to go with things.

Avatar Hover Height Project Viewer

[4:21]  Vir Linden is continuing to work on a couple of the bugs uncovered during testing. One of these is related to leg position issues, which can see an avatar’s leg “buckle” if the position is adjusted downwards such that the feet sink into the ground, becoming more and more pronounced the lower the avatar is set. Currently, there is no ETA on providing a fix for this.

Mesh Importer Viewer

[5:47 – via chat]  It is anticipated that the Mesh Importer project viewer (currently version ) will receive an update in week #10.

Tools Update Viewer

[6:56] A line has been drawn under the current set of changes to the viewer-side build tools, so the aim is now to get the viewer in the release channel.

However, there are further changes already being queued behind the current set of updates, many of them contributed by open-source developers, which the Lab believe will further improve the viewer build process. As such, it is likely that there will be further updates to the viewer in relation to the build tools, most likely in the form of a “build clean-up” viewer, which will likely include all of the agreed updates which didn’t make it into the current version of the build tools viewer code.

The hope is that, moving forward, all viewers – LL’s own and TPVs  – can, as far as possible, be built using the same tool set the only difference being whether or not a TPV sets a build switch to use proprietary libraries (e.g. Havok) or not.

Group Chat

[2:10] The Lab continues to roll-out updates as a result of this work, and updates focused on the issue of group chat servers themselves stalling, and they are confident that progress is being made. This is supported by anecdotal evidence from moderators of large groups with active chat, who are indicating the instances of visible server slow-down / stalling have been decreasing over the past week or so. Commenting on this during the TPVD meeting, Oz Linden said, “we haven’t declared victory in this yet, but it’s looking a lot more stable.”

In the meantime, some groups are reporting performance issues whereby for short periods – measured in terms of 2-3 minutes, where group chat messages won’t go through at all, and no time-out message is being displayed by the viewer, while other messages are being delayed, but then getting through, but without the massive roll of delayed messages. These may be related to previous changes made to improve the overall performance of group chat, and news of the problems is being taken back to the Lab for investigation.

Attachment Reliability

[4:39] As I most recently reported in week 8, there are on-going issues related to attachments. Vir Linden has also been investigating these, and there is a project viewer in development which includes fixes for those issues the Lab has been able to consistently reproduce. Unfortunately, the infamous BUG-6925 is not among these, but Vir hopes that the updates he’s been making will address it to some degree.

There is currently no ETA for the project viewer, but the Lab hopes that when it does appear, they’ll get some clear and constructive feedback on what it does and does not seem to address.

Continue reading “SL project updates week 9/2: TPV meeting, SL roadmap notes” →

SL project updates week 9/1: server, viewer

Posted on by Inara Pey
China Grove - blog post
China Grove – blog post

Server Deployments, Week 9

As always, please refer to the server deployment thread for the latest news and updates.

  • There was no deployment to the Main (SLS) channel on Tuesday, February 24th
  • On Wednesday, February 25th, all three RC channels will receive the same server maintenance package, which comprises:
    • A server-side fix for BUG-8297, “Unable to teleport anywhere using SLGO”
    • Improvement to server logging.

The SL Go teleport issue only affected users running the SL Viewer (SLV) supplied by OnLive, who run the SL Go service (it did not affect those using Firestorm on SL Go), and was the result of a recent server-side code clean-up which meant that OnLive’s SLV was essentially using an invalid channel name when instigating teleports, causing them to fail.

OnLive implemented a workaround for the issue on their side of things on February 3rd, however, the update included in the RC package noted above provides a fix on LL’s side of the connection as well.

SL Viewer

On Tuesday, February 24th, the Maintenance RC viewer, version 3.7.25.299021, was promoted to the de facto release viewer.  This release includes a large number of improvements and fixes to the viewer, as listed in the release notes.

This currently leaves just the new log-in viewer RC (version 3.7.25.298971 released on February 17th) and the Experience Tools RC (version 3.8.0.298001 last updated on January 15th) in the release channel for the time being.

Experience Keys / Tools

The initial release of Experience Keys / Tools is still awaiting various back-end updates to clear the Lab’s QA department, with the viewer remaining at RC status until that happens (see above). As previously noted in my Experience Keys reports, the initial release of the functionality will not support grid-wide experiences, although are likely to be on the roadmap for future enhancements.

Once released, the current iteration of the functionality is liable to be allowed to run for a number of months before the Lab starts making  (or releasing) updates, with Oz Linden commenting at the Simulator User Group meeting on Tuesday, February 24th, “there will be another round of improvements to Experiences at some point, but we’re going to give the first round some time to be adopted before we go too far with those.”

Group Chat

Group chat work is continuing, with the recent changes to handling server lock-ups reducing the number of reported freezes.

Simon Linden is also continuing to look at the code side of things, and specifically issues with group chat visibly “stalling” which switching between chat tabs and / or as a result of moving between regions. This work has been the subject of a series of “mini-tests” during the Server Beta User Group meetings held on Thursdays, and resulted in some recent code updates. Commenting on this at the SUG meeting, Simon said, “The new code also showed me details about one way the system fails, so I hope to make that better at recovering,” so further updates can likely be expected, although this may not be as broadly visible to many users as the initial group chat improvements.

SL updates updates week 8: viewer updates and general news

Posted on by Inara Pey

Soyokaze; Inara Pey, January 2015, on FlickrSoyokaze (Flickr) – blog post

Server Deployments

As will already be known, there were no server deployments this week.

SL Viewer

A new release candidate viewer appeared in the release channel on February 17th in the form of a new login viewer. Version 3.7.25.298971 sees the three log-in options buttons on the original log-in screen update replaced with a single button.

LL viewer log-in updates: as they first appeared after an initial log-in following the 2014 revisions to the log-in / splash screen (top); and as the log-in options are displayed in the new RC viewer (bottom) - click for full size, if required
LL viewer log-in updates: as they first appeared after an initial log-in following the 2014 revisions to the log-in / splash screen (top); and as they are now displayed in the new RC viewer, version 3.7.25.298971 (bottom) – click for full size, if required

February 17th also saw updates made to the Maintenance RC viewer, which was updated to version 3.7.25.298971, while the Tools update project viewer was updated to version 3.7.25.298862, having been held over for release from week #7, due to the Valentine’s weekend / President’s day no change window.

Group Chat

The was a further back-end update to the group chat servers during the week, further implementing the stability improvements the Lab has been working on to solve issues of chat appearing to freeze when switching between tab / issues with the chat servers themselves locking-up.

Other Items

Attachment Issues

As per my last report, there is still considerable debate about the nature of attachment issues, particularly those reported via BUG-6925. A concern is that while (as noted last time), the Lab considers there to be multiple issues occurring, BUG-6925 has already had some initial investigations carried out by the Lab, and these might be getting missed as the Lab considers the wider problems and attempts to categorise things. There is also a concern that the BUG-6925 issue in particular is AIS-v3 related, which could result in a significant increase in reported issues should that code be released by Firestorm as a part of their planned update.

Suggestions have been made on additional logging that might be added to the viewer to further pin-down precisely what is happening in the various circumstances where attachments are detached / re-attached or appear to be detached but aren’t, etc., and the issues are continuing to be examined by the Lab.

SL project updates 7/2: TPV Developer meeting

Posted on by Inara Pey
The Centaurs' Hall - blog post
The Centaurs’ Hall – blog post

The following notes are primarily taken from the TPV Developer meeting held on Friday, February 13th. A video of the TPV Developer meeting is included at the end of the article (my thanks as always to North for recording it and providing it for embedding), and any time stamps contained within the following text refer to both it and the TPV Developer meeting.

SL Viewer

Avatar Hover Height (AHH)

[09:17] It had been expected that the Avatar Hover Height project viewer (version 3.7.25.298129) might have been replaced by a release candidate during the course of the past week. However, some of the reported issues are still being worked on, most of which are related to the code which tries to keep an avatar’s feet in contact with the ground, and problems which can occur in certain extreme cases. It is now anticipated that the viewer will be updated to a release candidate viewer once these issues have been fixed.

 Viewer Build Tools

[14:37] It had been hoped that this viewer would be updated with a release candidate version prior to the long weekend no change window (see below), however it didn’t clear QA in time. Unless something significant happens on the QA side, this viewer should therefore reach release candidate status some time during week 8 (Week commencing Monday, February 16th).

However, it is anticipated it will undergo further updates while at RC status, and so therefore is unlikely to progress directly from RC to the de facto release viewer in a short space of time. One such update might be to include various patches to further simplify the build process, notably for windows, for example.

Graphics Settings and Avatar Rendering

[19:49] This work is a combination of STORM-2082, which allows users to create and save multiple graphics settings defaults, and internal work carried out by the lab to help users improve their viewer performance by defining limited above which avatars with heavy rendering costs are replaced by a solid colour. A viewer with these updates has been under limited testing for a while,  and is liable to be appearing very soon, particularly given the Lab’s recent announcement on forthcoming improvements.

Mesh Importer

[23:43] There are a fair number of bugs and issues which have been identified with the current mesh importer project viewer (see the JIRA filter list for details), it is therefore unlikely that it will be progressing in the short term.

Viewer-Managed Marketplace (VMM)

[00:05] A new version of the Viewer-Managed Marketplace project viewer was released on Friday, February 13th, version 3.7.25.298865. This viewer includes updates as a result of feedback from initial testing on Aditi, and is available for further testing, but again must be used on Aditi in regions ACME D, E, or F.

A number of issues remain open with This update:

  • VMM-20 Include product listing ID in audit listings window results
  • VMM-18 Add “sort by age of listing creation”
  • VMM-17 Support listing folders at any level of Marketplace Listings folder.

A list of key changes can be found on the forum update announcing the release of the new version of the viewer, and further details on VMM can be found in the SL wiki FAQ.

This viewer doesn’t address all of the concerns raised during testing, as some are still being discussed internally by the Lab, and a decision has yet to be taken on whether to address the additional concerns (which are presumably non-viewer specific) prior to holding a feedback meeting, or whether to push ahead with a further feedback meeting which might encompass the use of the new project viewer. This largely depends on the likely time frame for addressing the remaining issues: it might be as little as a week or so, or it might potentially be a month or so.

Experience Keys / Tools

[06:34] The Lab is continuing to work on back-end issues which need to be sorted prior to the project progressing. It is still believed that none of these issues directly affect the viewer or will require updates to the viewer, however, it is unlikely the viewer will be updated (other than to maintain parity with any other viewer that might be promoted ahead of it), until such time as the Lab is confident they are ready to start pushing things forward again.

Group Chat

The Lab is also continuing to work on group chat, and a further mini-test took place during the Server Beta User Group meeting on Thursday, February 12th. This focuses on testing viewer-side freezes when opening and closing or swapping between group chat tabs, and chat update times when moving across region boundaries either physically for via teleports. The overall results of these tests were that users experienced almost no viewer-side freezes in comparison to the last tests, where some severe viewer lock-ups were experienced.

[10:27] Most of the effort on group chat is now being directed into these issues of stability and eliminating the need for servers to be restarted as a result of server-side freezes, given that one the whole, performance for the majority (all but some of the very largest groups) has been noticeably improved. In terms of the server freezes, progress is being made, with Oz reporting that there might only be once such instance a day now as a result of fixes already implemented, which is currently being iterated upon as a result of  the additional monitoring / logging capabilities the Lab have also introduced.

Attachment Issues

[24:52] There have been increasing reports on variations of a problem with attachment rendering following teleports / region crossings (e.g. attachments seem to detach and then re-attach, people see their attachments as missing while other see them as still attached – and vice-versa, and so on). Some of the issues are listed in BUG-6925, and the issues have been noted by the Lab.

The likelihood is that these issues are the result of more than one problem, and possibly the result of various race conditions resulting from the complexities of data packaging and hand-over required in both a region crossing and a teleport (which are both essentially the same things – the packaging and handing of data relating to an avatar and its attachments between simulators, and then passing of updated information to users’ viewers). Some also appear to be directly related to AIS v3 (as per BUG-6925) Currently, the Lab is still prioritising SL issues and problems, and  full determination as to what needs to be down to resolve these problems has yet to be made.

In the meantime, if you do encounter the problem, and you’re preferably using the official viewer, you might want to consider filing the following information via BUG-6925:

  • The specific time, origin region (the place you were crossing  / teleporting from) and destination region (the place you were crossing / teleporting to)
  • Avatar name
  • The item ID (UUID) for the attachment(s) exhibiting a problem
  • The viewer log that was recorded at the time the problem occurred.

This will at least provide the Lab with data they can use in further investigations into the issues, once matters have been prioritised.

Other Items

Presidents’ Day

Monday, February 16th marks Presidents’ Day in the United States and Linden Lab will be observing the holiday. This means that there is effectively a no change window in force from February 13th through 16th inclusive. This in turn means:

  • There will be no Open-source development meeting on Monday, February 16th
  • Any server-side deployments scheduled for the week commencing Monday, February 16th will most likely take place a day later than usual. So any Main (SLS) channel deployment will probably occur on Wednesday, February 18th and any RC channel deployments on Thursday, February 19th.