Second Life project updates 26/1: server, avatar rendering

The DECADES event  - Saturday, June 27th, 2015 only - details here
The Decades event – Saturday, June 27th, 2015 only – details here

Server Deployments

  • There was no scheduled deployment to the Main (SLS) channel on Tuesday, June 23rd
  • On Wednesday, June 24th, all three  RC channels were updated with the same new server maintenance project, which included a fix for BUG-197, “Cannot See My Chat Only In My Region / Region Bad Performance” (not open to public viewing) and internal simulator fixes. As pointed out in the comments, this deploy was actually rolled back; I had forgotten to re-check the deployment page between originally drafting the first part of this article and publishing it.

The chat issue is a problem whereby a user can’t see any of their local chat on a region or parcel, and nor can anyone else due to a scripted object which is spamming chat so badly, the chat throttle kicks in, blocking their chat. However, no message would be provided to inform the user this was the case; with the change deployed on Tuesday, the user will now get a message about the chat throttle being hit, but unfortunately, the system will not identify the spammy object (so it might be removed, if you own it / the the rights to return it).

Avatar Complexity and Avatar Rendering in Busy Regions

Avatar Rendering in Busy Regions

During the Simulator User Group meeting on Tuesday, June 23rd, Simon Linden hinted that as well as the upcoming Avatar Complexity feature for which I recently gave a rapid overview, there are other options the Lab might consider in order to lighten the rendering load created by avatars:

We may experiment with a similar setting for crowds … setting a limit on the number of avatars we do any attempt at rendering.   In other words, if you were at a region with 75 people in view, and it was set for a limit of 64, you’d only get 64.  The remaining ones just wouldn’t be there in any form, similar to turning off avatars with ctrl-alt-shift-4.

He then went on:

That’s just an experimental idea now. To really make it better in a crowd, we’d probably want the server interest list to know and then it wouldn’t send you those updates.

As we’ve seen, the Interest list isn’t the easiest thing to play around with, so it’ll be interesting to see which, if either, of these ideas might be pursued.

Avatar Complexity

In terms of Avatar Complexity, questions have already been asked if the upper limit is adequate. With the test viewer, the Avatar Complexity slider runs from a value of 19,999 (which pretty much that no other avatars will render in your world view) means pretty much no other avatars will render in your field-of-view) through to 300K, above which sits “No Limit”, which means any avatar will render.

However, the suggestion has been made that the upper limit should perhaps be increased to allow for those who want to render particularly complex avatars used by friends. Responding to this, Oz Linden said, “It wouldn’t be hard to make the range somewhat wider, but at some point the control becomes too hard to use because each pixel is too big a jump.”

One issue that the new Avatar Complexity capability will not prevent (although, strictly speaking, it’s not designed to) is that it will not prevent worn mesh crashers impacting the viewer, because while the avatar is not actually rendered, the data on what is being worn still gets loaded into memory, and it is this that is used to crash things. Commenting on this, Simon linden said, “That sounds like something that should be looked at … if we can avoid loading that data, it would help everything.” Commenting on this, Oz Linden added:

There are a number of available optimisations; among them, using the complexity information from others to just pre-emptively not even fetch the appearance info for an avatar.

So again, it will be interesting to see what might come to pass in the future, should the Lab take this work up as well.

Second Life project updates week 25/1: server, viewer

Flux Sur Mer; Inara Pey, June 2015, on Flickr Flux Sur Mer (Flickr) – blog post

Server Deployments Week 25

As always, please refer to the server deployment thread for the latest updates / news.

  • On Tuesday, June 16th, the Main (SLS) channel received the server maintenance package previously deployed to the three RC channels, comprising further Internal server logging changes.
  • There will be no deployment or restart to the three RC channels on Wednesday, June 17th.

SL Viewer

On Friday, June 12th, the Lab issued a new viewer directly to release status. Version 3.7.29.302599 offers no functional changes to the the previous release viewer, but does include two DLL files – MSVCP100.DLL and MSVCR100.DLL which were missing from the Windows version of the viewer, and as a result causing problems for some users by their absence.

As a result of this release, the Attachment fixes RC viewer (Project Big Bird) RC viewer updated to version 3.7.31.302640 on Tuesday, June 16th, and the Experience Tools viewer updated to version 3.8.0.302622.

An “obsolete platform viewer” has also been issued by the Lab. Version 3.7.28.300847 of the viewer is a “static” release of the viewer which is aimed at providing users on Windows XP or versions of OS X below 10.7 to be able to continue to log-in to SL following the upgrade of the tools used to build the viewer. As I reported at the time, the key points to note about this viewer are:

  • It will not receive new features or bug fixes
  • It will not be promoted to release status
  • It does not change the Lab’s support policy on Windows XP or versions of OS X below 10.7, and is purely – as noted – an interim offering to help people
  • It will be provided for as long as is reasonable – but not indefinitely.

 Other Items

 Experience Keys  / Tools

As noted above, the Experience Tools RC viewer has been updated to match the current release version as the Lab continue to work on the back-end services. One issue that has been encountered  – albeit it in a single case so far – is that access to the KVP store used to hold information on the experience can be delayed in very permissive areas where there is a lot of natural in-world rezzing going on (e.g. public sandboxes) – see BUG-9027. This is because access to the store uses the same resources as used for rezzing objects.

The concern with the problem is that it could have an impact of grid wide experiences. However support for running experiences on a grid-wide basis is, in the Lab’s eyes, “still some time in the more distant future”, as they are focused on the region / parcel level and ensuring the capability works within these extremes first. As such, this issue is unlikely to prevent the initial deployment of the Experience Tools capability, although the Lab will look into this particular matter at a latter date.

Combat and Damage

During the Simulator User Group meeting on Tuesday, June 16th, the subject of damage, combat and protection within regions controls. Commenting on the matter Simon Linden said, “I’m beginning to think more and more we need regions set to different modes … something safe for general use, then raise the limits for combat (and the like) where you are freer to hurt yourself.  It’s just an idea at this point, but the one-setting-for-everything seems to always make someone unhappy.”

The problem here is that the existing Linden Damage system is seen as being somewhat inflexible, hence the development of various combat systems within SL. However, that there are so many systems now available, makes the Lab hesitant to change things, as Simon went on to explain:

We’ve talked before about working on the damage feature but I think that’s a case where everyone has their own usage, and so it would be better to have the features so others can make the damage systems they want. We probably can’t alter the current one. I’m sure somewhere somebody is using it and we can’t break their content. We could add to it, if it’s backwards compatible, but like I said, I think it would be better to hear what the CS and other system builders want to make their system nicer.

As Simon states, this doesn’t mean the Lab are about to make changes, or consider making changes where combat systems and damage are concerned; just that they are aware of the limitations within the current land settings.

That said, a recent change within the LL viewer has been noticed. Under it, damage cannot be set it the parcel level only; also, the viewer does not display the health meter on damage enabled parcels, but people can be “killed” and teleported home.  While Oz acknowledged this may be due to the Lab not fully following through on a set of changes to the viewer code, Simon also pointed out that the two-state on / off capabilities between regions and parcels has never really been fit for purpose:

Basically if you have any larger or high level setting, as well as a smaller scale (like regions vs parcels)  the higher one can’t just be “on” or “off”. It needs to be “on with override smaller settings” , “on without override”, “off override smaller settings” , “off without override” … It just doesn’t work with simple on/off settings.   You need more info about the intent on how it relates to the smaller areas

However, he again warned against anything being done on this in the near future, commenting, “if we want to fix this in SL, it means viewer UI changes, new data being passed back and forth to the simulator (which can be a hassle depending on the messages), new values in the database (which is another issue) and of course simulator changes.” In other words, were anything to be done, it would be a large-scale project, something which the Lab has yet to even consider taking on.

SL project updates week 24/1: server; VMM; group list changes

Goatswood; Inara Pey, June 2015, on Flickr Goatswood (Flickr) – blog post – visit soon, closes June 19th, 2015

Server Deployments Week 24

As always, please refer to the server deployment thread for the latest updates / news.

  • On Tuesday, June 9th, the Main (SLS) channel received the server maintenance package previously deployed to the three RC channels, comprising:
    • Change logic on accessing group member lists for large groups – please see more below
    • Internal server logging changes
  • On Wednesday, June 10th, the three RC channels should all receive a new server maintenance package comprising further Internal server logging changes.

Group Member List Changes

The “Change logic on accessing group member lists for large groups” refers to how the members list for groups with more than 5,000 members are now handled. A full explanation of the change and the reasons behind it can be found in my blog post on the matter. however, in short:

  • Groups with 5,000 or more members will no longer display the list of members unless:
    • You are assigned the Owner or Officer role within the group
    • You are assigned an ability within the group which requires the members list to be displayed (e.g. you are able to assign members to assigners roles, or are able to eject / ban people from the group, etc.)
  • Instead, and until corresponding changes are made to the viewer, all you will see on opening the members list as a message stating “Retrieving member list (0 / XXXXX)” – where XXXXX is the total number in the group,

The has already caused concern over how the change may be perceived as a functional breakage – see BUG-9393. In addition, two further issues resulting from the change have been reported:

  • BUG-9404: “Group members of large groups in a role which has “Invite people to this group” ability are not able to send group invites” (initially filed against the RC regions when the change was deployed in week #23, and now applicable to the grid as a whole)
  • BUG-9428: “Users using older viewers are unable to leave groups with >5k members on regions running 15.05.28.302161”

Scripting Memory Limits

A request was made for the Lab to consider allowing llSetMemoryLimit to request up to 128kb or 256kb of memory (whichever is more feasible), with a performance penalty for scripts using less than 50% of the memory requested – see BUG-9382.

The arguments for such an increase are not new; many coders run into problems with utilising memory for both code storage and code operation, resulting in having to write inefficient scripts and additional operations to communicate between scripts. A similar request has also been put forward (see BUG-8761), but which limits the additional memory allocation purely to Experiences on the ground that offering increased memory to all could lead to performance and other issues.

Commenting on the request at the simulator User Group meeting on Tuesday, June 9th, Simon Linden said:

I don’t think we’d want to limit performance … that seems like it would get into odd rules and conditions. Plus that’s likely to be in a place where we don’t want to add more code. FWIW, when you have lots and lots of scripts in a region, the time spent rotating through all the scripts becomes significant, so your script time isn’t just running scripts.

Oz Linden then added:

One of our frequent themes this year has been to look at various limits and consider making them better … perhaps we can look at the memory limit at some point too. One of the things I hope will happen as Experiences are adopted is that some of the code that’s being used to manage saving state and communicating can be replaced by simpler code to use the key/value store.

This drew agreement from Simon, who then continued:

I suspect larger script size has been limited by not having script memory limits. But at a smaller scale, it’s easy to add more scripts, so perhaps doubling or a bit more won’t really make it any easier to hog memory.

This doesn’t automatically mean that script limits will change in the future; as Simon also pointed out, script memory is the largest part of each avatar load, and can have an impact on things like physical region crossings and teleporting, which would likely have to be be considered. However, script memory is likely to get added to “The List” of things the Lab is looking at.

Viewer-Managed Marketplace

Some confusion has been evidenced over the use of the term “archived” in recent communications from the Commerce Team regarding what will happen during the auto migration process, and particularly with reference to items that have not seen sales in over a year.

  • The first point to remember is that any “archiving” will only occur for those merchants who have more than 5,000 items on the Marketplace when the auto-migration process reaches them. As noted in my last VMM update, all such affected merchants have already been notified
  • From information made available by those merchants so affected, it would appear that “archived” means “items will be returned to the merchant’s Received Items folder”. Firstly, any items the Merchant has stored on the Marketplace without any associated listing will be returned. If this fails to reduce their total count to below 5,000, then those items which have not seen sales for over a year will be unlisted and the items again returned to the merchant’s Received items folder.  From this it would seem that there will not be any actual “archiving” of listing information or items on the part of the Lab.

 

Second Life project updates 22/2; server and viewer

Obedience, LEA 1 - blog post
Obedience, LEA 1blog post

Server Deployments, Week 22 – Recap

The planned RC deployment scheduled for Wednesday, May 27th was rolled back as a result of a back-end issue. This currently leaves grid as a whole on the same server release.

Commenting on the roll-back at the Server Beta User Group (SBUG) meeting on Thursday, May 28th, Simon Linden said, “there was a minor issue but it was worth reverting; some internal tools weren’t running right and sending postcards was broken. [However] that code will likely be back next week, [as] I’ve already fixed the bug.”

These issues aren’t related to the region restart issues / caps failure people have noticed with some regions following a rolling restart, and as reported in my week 21/2 report, and which Simon indicates have yet to be looked into in-depth.

SL Viewer

Thursday, May 28th saw the Avatar Layer Limits viewer, version 3.7.29.301305, updated to the de facto release viewer. This viewer removed the limit of only being able to wear a maximum of 5 items per clothing layer (e.g. a maximum of 5 jackets and 5 shirts and 5 pants, etc), with a global limit of 60 layers which can be worn in any combination (e.g. you can wear 58 jacket layers, a tattoo layer and a pants layer if you wish).

This leaves two RCs in the release channel at present: the Avatar Attachment fixes RC (aka Project Big Bird and currently version 3.7.29.301943), and the Experience Keys viewer (currently version 3.8.0.300963, and which is awaiting the completion of back-end updates to the Experience Keys services). Both of these viewers will be updated to match the new release viewer, and it is anticipated that they will be joined by a new Snowstorm RC viewer in the near future (see below), which is currently awaiting some fixes prior to release.

General

Project news coming out of the Lab is a little light at the moment. This shouldn’t be taken to mean there isn’t a lot happening with Second Life. There are several projects that are in the pipeline – Viewer-Managed Marketplace and Experience Keys (/ Tools) being two that people are aware of.

The Lab don’t talk too much ahead of time as to what is going on, but it’s clear to see from Simon’s back-end work around avatar counts in regions, that there are various things which are being looked at. Again, we only recently had it confirmed that the Lab have, as a part of continuing work on improving the CDN services, shifted to another provider – and they are looking to move the delivery of more asset types to the CDN in the coming months.

In the meantime, we can expect to see more RC viewers appearing  – notably the next Snowstorm RC viewer with Avatar Complexity, and which should include STORM-2082, the ability to save and load graphics settings to assist with viewer performance, depending on the environment you’re in.

Jonathan Yap is working on the ability to various graphics settings in the official viewer, allowing users to quickly change between saved settings depending on their performance needs - this should be appearing in an upcoming Snowstorm contributions viewer (note the finished panel may not resemble the one shown left, above)
Jonathan Yap is working on the ability to various graphics settings in the official viewer, allowing users to quickly change between saved settings depending on their performance needs – this should be appearing in an upcoming Snowstorm contributions viewer (note the finished panel may not resemble the one shown left, above)

 

Second Life project updates 22/1; server, viewer, avatar rendering

Stand: Relay D'Alliez
Stand, Relay D’Alliez – Relay for Life Exhibit – blog post

Server Deployments, Week 22

Update, May 28th: a back-end issue with the RC deployment has meant that all three RC channels have been rolled-back to the their previous release, leaving the grid as a whole on the same server release.

As always, please refer to the server deployment thread for the latest updates / news.

There was no deployment to the Main (SLS) channel on Tuesday, May 26th, due to there having been no RC deployment in week #21.

On Wednesday, May 27th, all thee RCs should receive a new server maintenance package, comprising:

  • A change logic on accessing group member lists for large groups
  • Internal server logging changes.

SL Viewer

Due to Monday being Memorial Day in the United States, the Lab was closed for normal office business, and there was no meeting to discuss potential RC viewer promotions. However, the most recent update to the Attachment Fixes RC viewer (Project Big Bird, currently version 3.7.29.301943) is showing a must reduced crash rate compared to the previous release (and which prevented it from being promoted to the de facto release viewer).

The crash rate is still slightly high than for the current release viewer, but speaking at the Simulator User Group meeting on Tuesday, May 26th, Oz Linden described it as “probably not a statistically significant difference”. Whether this means the viewer will be promoted to release status later in the week or not, remains to be seen.

Increasing the Number of Avatars Per Region

Simon Linden: looking at ways an means to make it easier for the simulator and a viewer to better handle large numbers of avatars
Simon Linden: looking at ways an means to make it easier for the simulator and a viewer to better handle large numbers of avatars

“There’s one change that I will follow up on … I added a way so I can adjust the ‘max avatars in a region’ setting.  I’d like to do an experiment soon and see what falls apart if we can get over 100 people into a region,” Simon Linden said at the simulator UG when discussing the upcoming RC deployment.

“This is purely experimental and there are no plans for changing the SL limits,” he went on. “But sometimes regions hit 100 [and] it would be nice if the viewer and simulator handled that better.”

There is already an additional means en route to the viewer by which users can have greater control over how avatars around them in a region are rendered by the viewer, Avatar Complexity, when will draw avatars above a rendering limit set by the user as a solid colour (the so-called “Jelly Baby” avatars).  The will work alongside the existing Avatar Imposters capability already in the viewer.

However, in terms of his experiment, Simon suggested that one way to improve things might be for the viewer to simply not draw everyone within a region; although how this would work, and the criteria used to determine what avatars are drawn and which aren’t, does require careful consideration. Simon suggested the viewer might simply skip drawing those avatars that are furthest away once a threshold number of avatars in the region has been reached. Another (suggest by a meeting attendee) would be for the control to be via the Max Number of Avatars settings within the viewer – so that once exceeded, avatars are again simply not rendered.

As noted, Simon’s work is purely experimental, and primarily aimed at helping the Lab understand what might be done to improve things where there are large gatherings of avatars, and to perhaps try out one or two ideas based on what they learn.

Simon’s Rendering Tricks

As a part of the discussion on avatar rendering, Simon handed out a note card of tips and trick for improving your performance when dealing with complex avatars. While this includes the debugs which will form a part of the new Avatar Complexity functionality, which will be appearing in a a Snowstorm RC viewer soon, as well as suggestions which may already be known, I’m including his suggestions in full here for reference:

From Advanced > Show Debug Settings, set:

  • RenderAutoHideSurfaceAreaLimit   0
  • RenderAutoMuteByteLimit  0
  • RenderAutoMuteFunctions  7
  • RenderAutoMuteLogging  False
  • RenderAutoMuteRenderWeightLimit  350000
  • RenderAutoMuteSurfaceAreaLimit  150

In preferences / graphics, change “Max # of non-imposter avatars” to something like 8. Also try ctrl-alt-shift-4 to hide avatars, or ctrl-alt-shift-2 for alphas.

Note the two debugs shown in green are those related directly to Avatar Complexity and drawing avatars as “Jelly Babies”. Note that RenderAutoMuteFunctions must be set to 7 in order for this to work. Also note that the RenderAutoMuteRenderWeightLimit of 350,000 is purely an advisory starting point. The Lab estimate that this will reduce the very top 3% of very rendering-intensive avatars as solid colours. You may find you have to set the value somewhat lower in certain environments  – such as night clubs and dance venues – in order for it to be effective. I’ve personally found that 150-200K tends to be required in very busy ballrooms, etc.

Second Life project updates 21/2: general notes

Living in a Bowl
Living in a Bowl, May 2015 – blog post

Server Deployments, Week 21

As always, please refer to the server deployment thread for the latest updates / news.

On Tuesday, May 19th the Main (SLS) channel received the server maintenance package previously deployed to the three RC channel, comprising Internal server logging changes, back-end system bug fixes and a change to Reply-To e-mail addressing on snapshots. There were no RC deployments on Wednesday, May 20th.

SL Viewer

The Attachments Viewer RC (Project Big Bird) was updated to version 3.7.29.301943 on Thursday May 21st. As noted in part 1 of this week’s report, the initial RC release of this viewer had an elevated crash rate compared to the current release viewer, including a crash-on-exit bug, so this release will hopefully address those issues.

Group Chat

A fix for issues around BUG-9130, where some people were unable to see any posts in some or all of there group chats, including their own posts, while everyone else in the same group could see their posts, has started to be deployed across the chat servers, and should be completed on Friday, May 22nd.

“The chat servers got stuck with bad info about where the sender was, so the messages never reached them,” Simon Linden said at the Server Beta User Group meeting on Thursday, May 21st, reiterating an explanation given at a recent Simulator UG meeting. “And unfortunately it wouldn’t fix with relogging or even a chat server restart.”

“Loading…” Issue with Names in Group Chats

This is a viewer-side problem which causes avatar names to appear as “Loading” under certain circumstances in group chat (see BUG-3829 and STORM-2114). A contribution by Ansariel Hiller is currently with the Lab and is expected to be released as a part of the next Snowstorm contributions viewer, which is expected to appear soon.

Other Items

Region Restart Glitch

There has been something of a rise in reports of regions experiencing issues following recent following restarts – most noticeably caps failures. This is something the Lab is looking into, and Simon commented, “we have a suspicion that after rolls, as that server host starts up regions, it’s doing enough of them at about the same time that things get overloaded.   It’s still a theory but makes some sense why we’d get cap failures like that.”