SL project updates week 14/1: server and viewer updates, misc items

Piony Hideout, Lions Hill; Inara Pey, March 2015, on Flickr Piony Hideout, Lions Hill (Flickr) – blog post

Server Deployments

As always, please refer to the server deployment thread in the forums for the latest information and updates.

There was no deployment to the Main SLS channel on Tuesday, March 31st, due to the inventory issues arising from the week #13 RC deployment – see my update here for details.

On Wednesday, April 1st, all three RCs will receive the same update to the current server maintenance package. This update is specifically aimed at correcting the trash purging issue reported with BUG-8877. However, the fix does not address the issue of inventory fetching hanging if HTTP Inventory has been disabled within the viewer.

This is because the Lab regards UDP inventory fetching as a deprecated protocol path, as indicated by the release notes, and non-HTTP based inventory fetching is now being phased out. As such, it is anticipated the option to disable HTTP Inventory within the viewer is likely to be hidden / removed at some point in the future.

SL Viewer Updates

The Tools update RC viewer updates to version 3.7.27.300242 on Monday, March 30th, bringing it into line with the current release viewer (3.7.26 with Avatar Hover Height).

The Maintenance RC viewer updated to version 3.7.27.300323 on Tuesday, March 31st. also bringing it into line with the current release viewer,  and includes fixes for many of the bugs and issues encountered with the initial release of the RC.

Other Items

Diagonal Region Rendering Issues

A fair while ago now (late 2012 / early 2013 in fact), I reported on issues that had been noted with regions seeming to be “missing” when seen from other regions.The problem was originally reported in SVC-8130, which is still marked as “unresolved”, and it had been hoped that fix for SVC-8019 would address the problem as well as dealing with other issues. However, the problem has continued intermittently ever since, with numerous issues marked as duplicates of SVC-8130 being reported, with the issue most recently being seen when looking at regions diagonally opposite Brocade on the Mainland.

The return of the missing regions issue (if it ever really went away): looking north-east from Brocade towards where Mullein isn't on the Mainland. Not, as well, the region kitty-corner beyond Mullein is also absent the view, although both appear on the map
The return of the missing regions issue (if it ever really went away): looking north-east from Brocade towards where Mullein isn’t on the Mainland. Not, as well, the region kitty-corner beyond Mullein is also absent the view, although both appear on the map

The problems are regarded as handshaking / communications issues between region, and are generally resolved through a region restart; although understandably, some holding regions on Mainland are reluctant to request a restart as this can affect multiple other region as well.

Multiple Calling Cards

Most of us are familiar with calling cards in the viewer. They can be obtained by friending someone, or by someone giving you their card (or you giving them your card, and are useful for things like  opening people’s profiles from inventory (particularly handy in cases where you haven’t friended someone, and so don’t have to use search to locate them), or initiating an IM conference call.

One of the many calling cards issues - spawning multiple copies of the same card (images via Jessica Lyon)
One of the many calling cards issues – spawning multiple copies of the same card (images via Jessica Lyon)

However, within v3-style viewer they can also be annoying, as they have a tendency to multiple for no readily apparent reason. People can often have a set of calling cards under the Calling Cards folder, which can be partially replicated in the Friends sub-folder, and then fully replicated in the All sub-folder beneath that, for example. Or individual cards can get spawned multiple times across one or more folders for no readily apparent reason.

The problem here is that having a high load of calling cards can generate problems logging-in to SL, where they run out of curl multihandles, causing log-in to hang or for them to disconnect on logging-in. This can usually be solved through … wait for it … disabling HTTP Inventory and then logging-in and deleting them, but this may not be an option in the future (see the notes at the start of this report). Given that calling cards get re-spawned following a re-log after deletion, and can start multiplying again, Oz noted in the SUG meeting that they are now an area “worthy of some study”.

Rendering / Rezzing Failures and the CDN

In week #13 I reported on rendering / rezzing issues being experienced by some people in the Florida / Alabama region of the USA. Since that time, the Lab’s investigations through the CDN provider have suggested the the ISP in question (Mediacom) has degraded the service, possibly due to the volume of traffic coming from the CDN. This was a concern voiced early-on during the CDN implementation, but this appears to be the first time such a move has been confirmed. In the forum thread on the matter, which has seen more input on the situation, Monty Linden has offered some interim ideas that may help users experiencing problems, while also emphasising the Lab is still working with the CDN providers to further refine the service.

SL project updates week 13/1: server, RTLP and misc news

Server Deployments Week 13

As always, please refer to the deployment thread in the forums for the latest updates / news.

  • On Tuesday, March 24th, the Main (SLS) channel received the server maintenance package deployed to the three RCs in week 12, comprising updates which allow the Lab to make various configuration changes without having to necessarily run a rolling restart when they have done so. It contains not actual functional changes to the simulator software
  • On Wednesday, March 25th, the three RC channels should all receive the same new server maintenance package, which is focused on inventory loss issues, and provides the Lab with better error detection and logging, improving their ability to look at some of the failure places and the removal of unused code.

SL Viewer

On Tuesday, March 24th the Avatar Hover Height (AHH) viewer, version 3.7.26.299635 became the de facto release viewer. Avatar Hover Height is a new feature that allows you to adjust the vertical position of your avatar within some preset limits. This is a purely graphical tweak that does not affect your position for physics purposes. For it to work properly, both you and observers watching you need to be running a supported viewer.

You can find out more information view the wiki page and / or via my overview of AHH.

Now in the release viewer: Avatar Hover Height provides a means of adjusting your avatar's graphical height above the ground / floor / objects, as seen by yourself and others
Now in the release viewer: Avatar Hover Height provides a means of adjusting your avatar’s graphical height above the ground / floor / objects, as seen by yourself and others, on-the-fly

A very slight peculiarity with AHH, which seems to work very well, that if you have camera angle moved to the “default” view looking out from behind your avatar by hitting ESC to reset your camera angle, and use the AHH function, it can also change your camera angle. However this doesn’t happen if you’re using any other camera position at the time you alter your height using AHH. This appears to be because there is some interact between the avatar’s height and the default camera position which might be expected behaviour, and may be looked at again in the future. In the meantime, it doesn’t impinge on the overall functionality of AHH.

Restore to Last Position

Note that the RC update is does not include any deprecation of the server-side message used by the restore to last position code (RLTP) used by TPVs.

Commenting on the status of any removal of the server-side support for RTLP (see here for background on this) during the Simulator User Group meeting on Tuesday, March 24th, Simon Linden said, “We haven’t done anything about RTLP  and it’s still officially unsupported.   There’s a long list of issues that would really make that feature work.   It would be really nice, but it’s not just fix-one-bug.”

Oz Linden then added, “We won’t disable it completely without fair warning at the TPV meeting. What I’ve done so far is just ask questions – it doesn’t count as fair warning :-).”

 Other Items

Mesh Uploader

There is currently a mesh uploader project viewer (version 3.7.25.298441), which includes various improvements to the uploader, and which will most likely be progressing through to RC status and to a release status over time. However, there are still further requests for the uploader to be improved in terms of the information it displays, two of which are:

  • Better capabilities for zooming the model preview window after using the scale option, so that if the preview image is enlarged the user can zoom out further than is currently possible
  • The ability to provide an actual LI calculation while using the custom physics model upload, rather than just a convex hull measurement (see VWR-28177 “Enable Prim physics-shape-type physics weight display in upload floater”).

Both of these were raised at the Simulator User Group meeting on Tuesday March 24th; which doesn’t mean they’ll necessarily be acted upon, but while the Lab is tinkering with the uploader, it does bring both matters back to the Lab’s attention.

SL project updates week 12/1: server, RTLP, viewer translation tool

Tugby! Rugby with tugboats - blog post
Tugby! Rugby with tugboats – blog post

Server Deployments Week 12

As always, please refer to the server deployment thread for the latest updates and information on the week’s roll-outs.

  • On Tuesday, March 17th, the Main (SLS) channel received the server maintenance package previously deployed to the three RC channels.  This comprises “internal improvements for premium users”
  • On Wednesday, March 18th, all three RC channels should receive the same new server maintenance package, comprising various internal fixes for the simulator code.

Commenting on the RD deployment at the Simulator User Group meeting on Tuesday, March 16th, Simon Linden said, “the code going to RC tomorrow doesn’t have anything you will notice, but helps us when we want to make some configuration changes.” Oz Linden followed-up his comment by adding, “that is to say, it allows us to make changes we used to have to roll code to make.”

Other Items

Restore to Last Position (RTLP)

As noted in my last SL projects update report, the Lab is considering deprecating the last of the simulator messaging support for the “restore to last position” (RTLP) functionality, which can still be found in some TPVs and can, albeit with some undesirable results, depending upon how it is used, to restore an item directly from inventory  and return it to its last recorded in-world position, relative to the region in which the user is standing.

Because the Lab are considering the future of the simulator-side support for the capability, and also as I’ve reported, the Firestorm team is seeking use cases from users on how RTLP is useful, in the hope of presenting the Lab with a reasoned argument on why RTLP, or some similar capability should be retained / provided.

Oz Linden - keeping an eye on feedback through the Firestorm blog on "restore to last position"
Oz Linden – keeping an eye on feedback through the Firestorm blog on “restore to last position”

Commenting on the effort, Oz Linden said, “I’ve been following that. some are supportable, other not so much. Very good food for thought, though, and at least as far as I’ve gotten … people are heeding the request to be civil and non-hyperbolic. I really appreciated that.”

Also commenting on RTLP, Simon said, “What I’ve always wondered with that … the viewer really has a lot of info to do this better. It seems like it could record a list of things as they are removed from a region, then later put them back and give you better feedback on what works and what doesn’t.”

A number of possible approaches to handling cases currently managed by RTLP were also raised during the discussion, but these were just ideas, although they were listened to positively by Oz and Simon. However, this does necessarily mean the Lab will not deprecate the simulator support required for RTLP at some point in the future as a part of dealing with inventory loss issues, or that they’ll necessarily replace it with other functionality. But they are listening and reading.

So again, if you do have a good, concise use case for RTLP which hasn’t so far been reported on the Firestorm blog post’s comments (not here!), do make sure you hop over there and write it up.

Viewer Translation Tool Issue

Nalates Urriah reports that the translation option within the SL viewer (and used by TPVs) is effectively broken for those wishing to use it for the first time (see: BUG-8794 “The Bing API used by the viewer is depreciated [sic]”).

This isn’t an issue within the viewer per se; rather it is a result of the older Bing translation tool WPI / widget having been deprecated by Microsoft, and the means to access it removed from the web page to which users are directed on clicking the Bing API link within Preferences > Chat  > Translation in the viewer.

However, this isn’t a matter of the Lab simply correcting the link used within the viewer; deprecating the “old” translation service means that any keys obtained by following links to the “new” service will not verify through the API currently used by the viewer, thus preventing the Bing translation service being used.

Microsoft have deprecated the existing Bing translation API / widget with the result that new translation keys will not work within the translation function in the viewer (official and those TPVs using the Lab's translate options)
Microsoft have deprecated the existing Bing translation API / widget with the result that new translation keys will not work within the translation function in the viewer (official and those TPVs using the Lab’s translate options)

However, as noted in both the notes from Microsoft, and in comments accompanying BUG-8794, existing verification key previously obtained using the “old” Bing API are still supported, and will still work; thus, this issue only impacts those trying to obtain a new key. As this matter has just been reported, there is no word from the Lab on how it might be handled.

SL project news week 11/1: server, viewer, group chat

Armenelos, Calas Galadhon; Inara Pey, March 2015, on Flickr Armenelos, Calas Galadhon (Flickr) – blog post

Server Deployments Week 11

As always, please refer to the sever deployment thread for the latest updates and information.

  • There was no Main (SLS) channel deployment on Tuesday, March 10th
  • On Wednesday, March 11th, all three RC channels should receive the same new server maintenance package comprising “internal improvements for premium users”.

When asked during the Simulator User Group meeting on Tuesday, March 10th whether the “internal improvements for premium users” was related to the change to the in-world receipt of off-line IMs, as mentioned at the last SBUG meeting, Simon Linden could only say, “I’m not supposed to announce anything, so I can’t go into details … but one thing we’re looking at this year is ways to make premium accounts better. This may or may not do something like that eventually.”

SL Viewer Updates

A new Maintenance RC viewer, version 3.7.26.299610,was released on March 6th. This includes multiple fixes and improvements as listed on the release notes and download page.

The Experience Keys viewer updated to version 3.8.0.299338 on Monday, March 9th, maintaining parity with the current release viewer.

Experience Tools

Although the Experience Tools viewer has been updated (see above), there is still no news on when Experiences might be fully deployed. In order to help build interest in Experiences a suggestion has been put forward to enable Experiences to be rated in terms of the number of people actively joining them (see BUG-6911), which could be optionally shown (at the Experience creator’s discretion in things like search listings, allowing people to judge Experiences by their popularity.

The Lab has considered allowing users to rate Experience themselves in a future update – but as point out in the JIRA comments, such a system could be open to gaming, much like the old avatar popularity ratings. BUG-6911 has been imported by the Lab, but it is currently unclear if the idea will be carried forward.

Group Chat

As also noted in my last updates, recent changes to the group chat service have seen up to a 20% failure rate in delivered messages. Simon Linden spent a fair amount of time during week #10 stabilising things once more, and notes that the situation taught the Lab more about how things might fail. He currently has a set of updates which may further improve things, and these are liable to be tested at the next Server Beta User Group meeting.

Other Items

Names Vanishing from Ban Lists

There have been reports of avatars added to a region / estate ban list or have been previously muted suddenly dropped from the list without an action on the part of the list owner. This might be connected to the old issue of bans made using radar on some older versions of v1-style viewers (notably Phoenix) failing to “stick”, or it may be something else, such as a failure to correctly update a ban / mute list.

Commenting on the subject at the Simulator User Group meeting, Simon said, “we’ve heard reports of that and have looked into it … if you ever can narrow down an instance of that happening, please note it in a JIRA … Our logs will record info about those changing but we have to know where and when to look

“I’m making wild guesses, but I think it would be either the viewer or the simulator making an update to the ban list, and somehow having bad data.   Perhaps an incomplete list gets into the picture, and using that as a basis for the update it drops people.   Our logs will show events like “MrNoisy was added” and “MrGoodBehaviour was removed” but finding the event is the missing part of the puzzle.

“If you have multiple regions in the estate, there’s another issue of having the changes sent out to all the regions.   We’ve seen failures there and I know it’s been worked on a few times (and suspected in some of these reports).”

So, if you do encounter a situation involving an apparent ban list failure, and can log the exact circumstances / details, please consider raising a bug report.

SL project updates week 10/2: server and viewer notes

RMS Titanic, Second Life
RMS Titanic, Second Life

Server Deployments

Tuesday, March 3rd, saw the Main (SLS) channel receive the server maintenance package deployed to the RC channels in week #9. This includes:

  • A server-side fix for BUG-8297, “Unable to teleport anywhere using SLGO”
  • Improvements to server logging.

There were no deployments to the RC channels on Wednesday, March 4th.

Upcoming Server Updates

During the Server Beta User Group meeting on Thursday, March 5th, Caleb Linden, who now chairs the meetings, indicated that an upcoming deployment to the RC channel will include a change to the number of off-line IMs delivered when you log-in.

Currently, the cap is set to 25, but it’s not clear what the new limit will be, although it might vary (or be adjustable) depending on whether a user is a Basic or Premium member. This change will not affect the forwarding of off-line IMs to e-mail.

SL Viewer

The new Tool Chain Update viewer was promoted to Release Candidate status on Wednesday, March 4th. Version 3.7.26.299443 see a change to the tools used to build the viewer (e.g. the use of Visual Studio 2013, Xcode 6.1, and some other tools improvements). This update also incorporates the log-in screen update to a single button, replacing the need for the Log-in Screen RC viewer.

Future Viewer Updates

Avatar Layers Global Limit

In 2014, BUG-6258, “Popularity of Mesh Attachments Facilitates Need For More Alpha Layers” is a request to raise the number of alpha layers which can be concurrently worn, prompted the Lab to consider the introduction of a global limit on the number of clothing layers an avatar could wear,so that, like attachments, an avatar can wear as many clothing layers in any combination, up to that limit, rather than having individual limits enforced per clothing layer.

During the TPV Developer meeting in week #9, Oz Linden indicated that while there had been major progress on this idea since it had been first raised, something would likely be appearing later in the year. Since then. Whirly Fizzle has spotted a viewer code repository from Vir Linden: “SL-103 WIP – viewer enforces wearable limits based on total articles of clothing rather than per-type limit. Limit is artificially low for testing, will bump before release.”

This doesn’t mean new global limits on avatar cloting layers will be appearing in the next few weeks, but it does indicate that it is one of several avatar related projects he’s working on (together with the new Avatar Hover Height feature and fixing at least some of the current crop of attachment issues).

Notifications Changes

Also during the TPVD meeting in week #9, Oz commented that some “cosmetic” work was being done to the way notifications are presented through the viewer. Again, thanks to Whirly’s poking around, this would appear to be in connection with BUG-8000, “Separate transaction notices from group notice/invites or provide some means for sorting incoming notifications”, which has been imported by the Lab and is being worked on.

A preliminary version of the proposed new notification panel (see BUG-8000). Note that is is a work-in-progress and may be subject to change prior to release (image courtesy of Whirly Fizzle)
A preliminary version of the proposed new notification panel (see BUG-8000). Note that is is a work-in-progress and may be subject to change prior to release (image courtesy of Whirly Fizzle)

 Group Chat

Simon Linden is continuing to bang on things to try to correct issues that have arisen since the last set of updates at the end of week #9. According to Caleb Linden, who now chairs the Server Beta Group meetings, those groups that are affected are seeing up to a 20% failure rate in delivered messages.

Other Items

Rigged Mesh Crashers

Further to my last report on this issue, Whirly Fizzle (who is genuinely one of the great powerhouses in helping to keep SL rolling along) stepped up to provide a more detailed explanation of the issue based on her own extensive investigations into the matter (thanks Whirly! 🙂 ).

In her comments, Whirly notes that Chalice Yao has developed a debug  setting within her own NACL viewer setting specifically stop cap rendering of worn rigged meshes, and that Nicky Dasmijn from the Firestorm team is working on a patch for their viewer. However, while the setting works, it can also prevent “legitimate” rigged mesh items with high vertices counts from rendering.

Further discussion of the problem with regards to BUG-2082, which includes new viewer rendering controls, confirmed this would not help with the issues presented by this particular kind of rigged mesh crasher but it will prevent issues such as the animated mesh “tail of doom” which can cause viewer crashes as a result of overloading memory.

SL project updates week 10/1: server, general news

Leka, Nordan om Jorden; Inara Pey, March 2015, on Flickr Leka, Nordan om Jorden (Flickr) – blog post

Server Deployments

Tuesday, March 3rd, saw the Main (SLS) channel receive the server maintenance package deployed to the RC channels in week #9. This includes:

  • A server-side fix for BUG-8297, “Unable to teleport anywhere using SLGO”
  • Improvements to server logging.

There were no scheduled deployments to the RC channels on Wednesday, March 4th.

Group Chat

Following the last deployment of back-end group chat changed during week #9, some large groups with active group chat have reported an increase in issues of message failures, although they appear to do so somewhat randomly, with some people seeing them and others simply not receiving them at all.

Commenting on the problem at the Simulator User Group meeting on Tuesday, March 3rd, Simon Linden summarised the situation thus:

In short, yes, it’s cranky, and yes, we’re (as in I am) looking at it … the chat server itself is actually running better than before, believe it or not. A back-end service it relies on, what we call “agent presence” [used to help locate someone on the grid], seems to be having new problems, so the changes may have added load to those servers and is causing problems, or something else unexpectedly changed … [So] some people don’t get the messages when chat is failing … it’s dropping sending some updates and messages when it times out with some other internal requests.

Further updates will be provided as the Lab / Simon continues to look at the problems.

CDN Notes

There have been recent reports of people experiencing slow texture and mesh load issues, leading to questions concerning the CDN service (although some of the issues that have been mentioned might be related to local caching more than the CDN). In particular questions have been asked as to how long a CDN server retained its cache of data relating to regions prior to going “cold” and requiring a “reload” from the SL services. Commenting on this at the Open-source Developer meeting on Monday, March 2nd, Oz Linden said that some CDN caches do age out more quickly than others.

The Lab has also been experimenting with more than one CDN provider, and are continuing with different CDN configurations as well to further tune things, as well as continuing to measure results; so we may yet see further changes  / improvements, and a possible decrease in instances that may be related to “cold” CDN loads.

Other Items

Rigged Mesh Crashers

The Server Beta Meeting on Thursday, February 26th saw the issue of a “new” mesh crasher being used on the grid. This is essentially a deliberately corrupted rigged mesh attachment which, when worn will cause viewers around it to immediately crash, with no warning or ability to take preventative action, such as muting the offending avatar.

Just over a year ago, some advice was given on how to counter graphics crashers by adjusting the viewer’s debug settings, and some people many be getting pointed towards it again in order to avoid being affected by the “new” crasher.

However, changing the specified debug settings can lead to a failure to render much of what you actually want to see, as noted in  this comment following the article. At the time the advice was given, the Firestorm team tracked many of the problems their users were experiencing directly to the settings having been changed. Ergo, if you are pointed to this particular article as a means of combating graphics crashers, please keep in mind you may gain undesirable results, and keep a note of the original settings so you can switch back to them should this be the case.

During the discussion on this matter at the SBUG meeting, speculation was raised on whether or not the forthcoming new viewer rendering controls (see: STORM-2082). opinion is divided, as the viewer downloads the data which may cause a graphic crash and starts processing some of it in order to determine what to render or not, and even this initial processing could be enough to crash it.

SL Feed Issues

There has been an uptick in the number of snapshot uploads to the SL feeds failing over the course of the last week, with some additionally reporting issues of comments failure to appear / “loves” failing to stick. Some users also reported issues over the weekend with web profiles failing to load, and a JIRA (see BUG-8677) was logged on this issue on March 3rd.

The last several days have seen people again encounter issues with snapshots failing to process / display in their feeds
The last several days have seen people again encounter issues with snapshots failing to process / display in their feeds

Whether the two issues had a common cause isn’t clear, but as the latter has been resolved, and you are one of those continuing to experience snapshot upload failures, please file a JIRA providing as much information as possible (links to any feed post with a missing snapshot, date / time of upload, number of failures, etc.).