SL projects update week 31 (1): server releases, SSA, Interest List

Update, July 31st: further to the note below relating to the BlueSteel RC deployment, it appears the bug fix did not clear QA in time for the deployment to occur. Maestro Linden has updated the deployment thread to read: “BlueSteel’s planned project hit some last minute issues, so the update has been canceled for this week.  Instead, BlueSteel will not be rolled this week (it will match the version on the ‘Second Life Server’ channel).

Server Deployments Week 31 (Week Commencing Monday July 29th)

As always, please refer to the week’s forum deployment thread for news, updates and feedback.

Second Life Server (SLS Main) Channel

On Tuesday July 30th, the SLS Main channel received the server maintenance package previously deployed to BlueSteel in week 30 and LeTigre in week 29. This project includes:

  • A further fix for the issue of pathfinding characters using CHARACTER_STAY_WITHIN_PARCEL getting stuck if they somehow exited their home parcel
  • Fixes for objects failing to detect collisions after teleporting (BUG-969) and run time permissions failing to function correctly on attachments (BUG-2931)
  • New capabilities to the materials system to better handle texture requests.

Release Candidate Channels – Wednesday July 31st

Magnum and LeTigre will remain SSA enabled and both receive the updates deployed to the Main channel.

BlueSteel should receive a new server maintenance project. However, a last-minute bug was found in the code. While this has been fixed by Kelly Linden, it has still to pass LL’s QA at the time of writing. Assuming the package passes QA and is deployed, it will include:

  • Fixes for some simulator crash modes
  • A fix for BUG-3291 (“llListen in linked objects is listening at root instead of linked object local position *after re-rezzing the linkset*.”)
  • A fix for BUG-3307 “(llApplyImpulse called from attachment does not work on avatar if script is reset or started when attached”).
The Simulator UG meeting, Tuesday July 30th.
Doyouthinkhesaurus – Baker Linden (far right), in his new avatar look, literally towers over the start of the Simulator UG meeting on Tuesday July 30th.

Server-side Appearance

As noted above and in part 2 of last week’s report, SSA will not be enabled on any additional server channels this week, but does remain enabled on LeTigre and Magnum as the Lab continue to gather statistics and monitor performance, etc. Commenting on the state-of play, Nyx Linden said, “Testing seems to be going well, but we’re being on the cautious side – making sure that the back-end can handle the load. There are a few reported bugs in JIRA, but *most* are minor and we’re working on the not-so-minor ones.”

Issues

I provided an update on some of the more serious issues the Lab has been addressing with SSA in part 2 of my week 30 report (see the link in the paragraph above). Since then I’ve been poked about additional advice the Firestorm team have put together for those experiencing issues, including SUN-98,and I’m providing the relevant information here.

Avatar textures remaining grey / SUN-98: is generally the result of wearing a corrupted clothing asset, and as such is “expected behaviour” in order to avoid cases of accidental nudity, as I explained last week. To help diagnose the problem, the Firestorm team suggest you:

  • Remove all clothing and allow the avatar to bake with just the skin layer worn. If it fails to bake properly, the skin is the corrupted asset and needs to be replaced
  • If the skin bakes correctly, start adding the clothing layers of the outfit one at a time and check each to see how it bakes
  • If an item shows-up fully or partially grey, that is the corrupted asset. Replacing it should allow everything to bake and render correctly.

The Firestorm article also includes some Firestorm-specific actions for problems, and is a work-in-progress, so you can refer to it via the link above for further advice.

Continue reading “SL projects update week 31 (1): server releases, SSA, Interest List”

SL projects update week 30 (2): Upcoming server & viewer releases, SSA, HTTP

Server Deployments Week 31 (Week Commencing Monday July 29th)

As always, please refer to the week’s forum deployment thread for news, updates and feedback.

Second Life Server (SLS Main) Channel

On Tuesday July 30th, the SLS Main channel should receive the server maintenance package previously deployed to BlueSteel in week 30. This project fixes some miscellaneous bugs, and also allows viewers to send requests for materials data more rapidly.

On Wednesday July 31st, the three main Release Candidate channels should be updated as follows:

  • BlueSteel should receive a new server maintenance project.  This project fixes some bugs related to LSL scripts in child prims of linksets, and also addresses some server crash modes
  • Magnum and LeTigre remain SSA enabled and both receive the updates deployed to the Main channel.

Server-side Appearance

As noted in the planned deployment summary above, it is currently not anticipated that SSA will be enabled on any additional channels in week 31.

Overall, the Lab think the initial phase of deployment is going well, and recognise the considerable contribution made by TPVs in enabling this to happen. A rough approximation from viewer statistics suggests that around three-quarters of users logging-in to SL are using viewers which are SSA-enabled, and that the overall figure may be higher.

A chart compiled by Kadah Koba showing the percentages of SSA-enabled and non-SSA viewer in use (excluding Firestorm 4.4.0)
A chart compiled by Kadah Coba showing the percentages of SSA-enabled and non-SSA viewer in use (excluding Firestorm 4.4.0)

Commenting on the state of play for the project during the TPV Developer meeting on Friday July 26th, Nyx Linden said:

The system is working pretty much as we expected … and even the scaling of how much load is being generated is pretty much right on par with what we’re expecting. But we want to make sure that a few other things are returning the right things and we’re getting the right statistics that we want before we roll it out to the [entire] grid. We’re trying to be extra-cautious.

Viewer-side Updates

In terms of viewer-side updates, the plan is to try to have one major post-SSA enabling release which should include the planned inventory updates noted in the first part of this report along with any additional viewer-side code tweaks to the viewer arising from SSA being enabled, and a final code clean-up to remove the “old” baking code.

However, this does depend on enabling SSA on the rest of the grid. If there is yet cause to delay this (due to an unexpected issue arising, for example), and the delay continues for a significant amount of time, then it is possible that there will be two viewer releases: one with the currently planned updates and one with the post-deployment code clean-up.

Either way, to assist TPVs prepare for the viewer-side update(s), Nyx plans to periodically push code from the Lab’s private repositories to their public repositories as and when code is in a suitable condition to be pushed.

Issues Update

SUN-98 (Bake fail resulting from partially broken alpha layer): this is thought to be the result of wearing a corrupted clothing layer, and if so is considered to be expected behaviour in order to avoid cases of “accidental nudity” (which might arise from wearing a corrupted clothing later, which the SSA system would ignore and just bake whatever was underneath it  – such as the avatar’s skin). However the matter is still being looked into in case the problem has another cause.

Nyx acknowledged that even if the problem is due to expected behaviour, it would be useful  “at some point in the future” to add some UI elements to actually show the user which clothing asset they’re wearing that is causing the problem. What form these UI elements / warning will take remains to be decided.

SUN-99 (Bakefail on SSA regions only. When entering into SSA region, skin and system clothes fail to bake): this issue only affects a very small number of users and appears to be related to them having multiple copies of the Current Outfit Folder (COF) in their inventories, probably as a result of having moved it  within their inventory (i.e. into another folder) at some point prior to the Lab introducing restrictions to prevent the COF being moved or deleted.

To prevent this happening in the future, the Lab is implementing further back-end restrictions and other improvements on the COF, and Nyx has e-mailed all TPVs with notes on how the COF should be implemented within the viewer in order to comply with these restrictions.

In the meantime it was mentioned at the Server Beta meeting on Thursday July 25th that LL’s support team can now assist users who find they are suffering from this particular issue.

Viewer Updates

Release Candidates

As noted in part one of this report, there are now three RC viewers in the viewer release channel (Beta Maintenance, Google Breakapad and Vivox). All three are performing well, although no decision has been made as to which will be going to release status first.

Beyond these, the Lab is looking at a number of further release candidate cohorts, including the Cocoa updates for the Mac version of the viewer, a series of open-source contributions to the viewer, and a further series of CHUI updates.

Commenting on the current situation with viewer updates at the TPV Developer meeting, Oz Linden said, ” It’s going to be some time before we get to the point where we’ve got the number of simultaneous things happening down to a reasonable number; lots of stuffing was sitting around waiting for the opportunity to get out, and it’s all coming at once now!”

Continue reading “SL projects update week 30 (2): Upcoming server & viewer releases, SSA, HTTP”

SL projects update week 30 (1): server releases, viewer, SSB/A

Server Deployments – Week 30

As always, please refer to the week’s forum deployment thread for news, updates and feedback.

Second Life Server (SLS Main) Channel

On Tuesday 23rd July, the SLS Main channel received the server maintenance package previously deployed to BlueSteel. This comprised a further package of under-the-hood changes related to the experience tools.

Release Candidate Channels

On Wednesday July 24th, the three main Release Candidate channels should receive the following updates:

  • BlueSteel should receive the same server maintenance project that was on LeTigre in week 29, and which additionally includes the experience tools updates deployed to the Main channel
  • Magnum and LeTigre should both see Server-side baking / appearance (SSB/A) enabled, and should both receive the experience tools updates deployed to the Main channel.

Viewer Updates and Release Process

The second release candidate viewer was made available on Friday July 19th. Version 3.6.2.278609 comprises the long-awaited Vivox updates. This was followed on Monday July 22nd by the third release candidate, version 3.6.2.278615, which contains Google Breakpad updates.

Commenting on the first two release candidates to be deployed (the Beta Maintenance RC and the Vivox RC), Oz Linden said at the Open-source Dev meeting on Monday July 22nd that, “they each got as many users as we asked for, and we’re getting good data on them.” However, this doesn’t mean that either one will is likely to become the de facto release viewer yet, as Oz went on to note, “we configured both of these for a relatively small number of users just in case… we might want to raise it before we make a release decision.” Given that the Google Breakpad RC has been added to the mix, any decision on which get promoted to release status may well be held over even longer as numbers are crunched.

Release candidates are now also listed on the Official Alternate Viewer wiki page, where they can be downloaded manually. In light of this, I’ve updated my overview of the new viewer release process to include notes on manually downloading and installing release candidate viewers.

Server-side Baking / Appearance

As noted above, following the RC channel restarts due on Wednesday July 24th, both Magnum and LeTigre should be running with SSB/A enabled. Overall, the response to SSB/A deployment both on LeTigre (week 28) and Magnum (week 29) has been good, with few issues being reported.

Of those which have, some may be tied to the way in which some TPVs have implemented the Current Outfit Folder (COF). To help determine whether this is the case, Nyx Linden issued an e-mail on Monday July 22nd, outlining how the Lab anticipates the COF should be set-up within a viewer, and has asked all TPVs to verify that they’ve met the requirements.

More on COF Mismatch Issues

In week 29, I referred to the issue of COF Mismatch Issues. These tend to occur when your viewer and the baking service disagree on the COF version number on which your appearance should be based, resulting in “COF version mismatch” errors appearing in the viewer. Part of the problem is due to the inventory protocol relying on both HTTP and UDP messages, some of which have failure callbacks and some which the viewer may wrongly assumes completes successfully – and the “COF version mismatch” results.

To eliminate this, the Lab is working to update the Agent Inventory Services (AIS), which will see the most error-prone operations related to the COF converted to use AIS rather than UDP. The hope is that this work will both remove the most prominent causes of COF mismatch errors and reduce the number of network calls needed to update the COF. This work has been ongoing for a while, and will form part of the next phase of SSB/A work once the current deployment has seen SSB/A go grid-wide. These updates will involve further viewer-side updates, and include a range of additional improvements, although as yet there is no time scale for their release (particularly as the Lab is only just starting discussing them with TPVs).

Group Ban List

There is not a lot to report here. Baker Linden is still working on the viewer-side code. Giving a brief update at the Simulator User Group meeting, he said, “I’m currently deciding on the format of the data coming into the viewer, and adding it to the group manager subsystem in the viewer. That’s about it :).”

Related Links

SL projects update week 29 (2): server, viewer general news

Server Deployments week 29

As always, please refer to the week’s forum deployment thread for news, updates and feedback.

On Tuesday July 16th, the SLS Main channel received the server maintenance package previously deployed to Magnum in week 28.

On Wednesday July 17th, the three main Release Candidate channels received the following updates: individual updates:

  • Magnum became the RC with the server-side baking / appearance project enabled (SSB/A is disabled on LeTigre). This move was made to expose SSB/A to a larger number of regions and a larger number of users as a result
  • BlueSteel received a further package of under-the-hood changes related to the experience tools
  • LeTigre received a new server maintenance project, which included fixes for several issues, including a further updated for pathfinding characters using CHARACTER_STAY_WITHIN_PARCEL getting stuck if they somehow exited their home parcel. It also added “RenderMaterialsCapability” to the /simulator/features cap, which indicates the access rate allowed when accessing the “RenderMaterials” capability, and Increased the “RenderMaterials” capability access rate to 4 requests per second (up from 1).

SL Viewer Updates

Release Candidate Viewers & the Release Process

The first of the viewer Release Candidates became public on Thursday July 19th. This is a maintenance update (3.6.2.278602) with a number of individual fixes from LL’s viewer maintenance team.

While the actual order of release is not clear, it appears that the next Release Candidates which will be added to the viewer release channel will be:

  • Viewer Breakpad changes
  • The Vivox updates
  • A Snowstorm (code contributions) build.

Which of these RCs is promoted to be the defacto release viewer will be depend upon a number of factors, including how well each performs as a release candidate (in terms of performance, crash rate, etc.).

Because Release Candidates are “cohorts” within the viewer release channel, you cannot download them as a distinct viewer installer package via the Official Alternate Viewers wiki page. However, release candidates can be tracked (and the source code obtained by those interested in self-compiling viewers) from the Official Viewer Source Repository page.

To give some idea as to why the new process has been introduced, Simon Linden indicated there is around four months of viewer work currently backed-up and awaiting release. This may mean that initially, there might be a higher bumber of RC viewer cohorts in the release channel than will be the case once the backlog starts to clear.

For a complete breakdown on how the new release process works, please refer to my explanation of the process.

I’ll be endeavouring to keep pace with official viewer updates, including cohorts, through my Viewer Round-up Page.

Settings.xml

The plan to use a single settings.xml file for all installed versions of the SL viewer is currently dependent upon a snowstorm code contribution which is currently in the pipeline. Once this has been implemented within the viewer code, it should help eliminate problems of users on the SL viewer reporting their settings have been “eaten” / overwritten should they move between different versions of the official viewer (i.e. swapping between the release viewer and a project viewer and back again).

Group Ban List

Baker Linden continues to work on the group ban list project (see JIRA VWR-29337). The last time he was available for an update at the Simulator User Group on Tuesday July 10th, he indicated that he was working on the back-end code, which required a lot of refactoring and which he was hoping to get finished by the end of week 28 prior to moving to the viewer-side code.

Speaking on his behalf at the Server Beta meeting on Thursday July 18th, Simon indicated that Baker may have achieved his goal, and is now working on the viewer side of things and the UI.

Experience Tools

Again, no major news here. The server-side updates (which presumably include the long-awaited permissions system updates) continue to reach RC channels, but there is no news on the viewer-side updates which are expected to be appearing in a project viewer at some point in the (hopefully) near future.

Related Links

SL projects update week 29 (1): server, viewer, SSB/A

Server Deployments – Week 29

As always, please refer to the week’s forum deployment thread for news, updates and feedback.

Second Life Server (SLS Main) Channel

On Tuesday 16th July, the SLS Main channel received the server maintenance package previously deployed to Magnum in week 28, intended to fix a couple of pathfinding issues:

A slight issue at the start of the rolling restart process on Tuesday 16th July meant that some regions on the main channel experienced to restarts, with the second updating to the correct release.

Release Candidate Channels

On Wednesday July 17th, the three main Release Candidate channels should each receive individual updates, as follows:

  • Magnum will become the RC with the server-side baking / appearance project enabled
  • BlueSteel should receive a further package of under-the-hood changes related to the experience tools
  • LeTigre should receive a new server maintenance project, which includes the following fixes and new features:
    • Fixes:
      • BUG-969 “teleporting breaks collision detection state for volumedetect objects”
      • BUG-2931 “run_time_permissions no longer triggers in attachments after requesting 0 permissions”
      • A further fix for the issue of pathfinding characters using CHARACTER_STAY_WITHIN_PARCEL getting stuck if they somehow exited their home parcel
    • New Features:
      • Added “RenderMaterialsCapability” to /simulator/features cap, which indicates the access rate allowed when accessing the “RenderMaterials” capability
      • Increased the “RenderMaterials” capability access rate to 4 requests per second (up from 1)

Viewer Updates and Release Process

As I reported at the time (see New viewer release process implemented), the new viewer release process went live in week 28. I’ve provided a complete breakdown of the process and what it means in general, for those who wish to know more.

This has seen a number of beta and project viewers appear on the revised Official Alternate Viewer wiki page, with updated viewers including:

  • On July 15th the Second Life Beta channel saw a new release  – version 3.6.2.278491 (release notes)
  • CHUI updates continue to appear first in the CHUI project viewer, which released version 3.6.2.278372 on July 9th
  • The project Cocoa viewer for the Mac also updated on July 15th, to version 3.6.1.278025.

SSB/A Update

In a late change to the deployment schedule, Magnum will the RC channel to have SSB/A enabled following the rolling restarts on Wednesday July 17th.

This will include a fix for BUG-3203, the “notecard bug” I reported on in week 28 (with thanks to Whirly Fizzle), wherein  if you create a notecard in an SSB/A region (i.e. a region on the LeTigre RC at the moment) and attempt to embed anything in it (e.g. LMs, textures, other notecards), the notecard will fail to save with the message: Unable to upload (asset ID number) due to the following reason: The server is experiencing unexpected difficulties. Please try again later.

Continue reading “SL projects update week 29 (1): server, viewer, SSB/A”

SL projects update week 28 (2): server, SSB/A update

Server Deployments – Week 28

As always, please refer to the week’s forum deployment thread for news, updates and feedback.

On Tuesday 9th July, the SLS Main channel received the server maintenance package previously deployed to all three RC channels in week 26.

On Wednesday 10th July, the three RC channels received the following packages:

  • LeTigre had Server-side Baking / Appearance code enabled. See my mini-update here
  • BlueSteel received a further package of under-the-hood changes related to the experience tools
  • Magnum received a server maintenance project intended to fix a couple of pathfinding issues:

Week 29 Deployments

While the final arrangements for the week 29 deployments (week commencing Monday July 15th) have yet to be finalised, Maestro Linden indicated at the Server Beta meeting on Thursday July 11th that there is likely to be one new server maintenance package going to an RC channel during that week, This is likely to have:

  • Further fixes for various crash modes
  • An additional update for pathfinding characters with CHARACTER_STAY_WITHIN_PARCEL=TRUE, which should enable such characters to re-enter their assigned parcel should they accidentally exit it (as a result of being pushed out, for example), rather than getting stuck
  • Fixes for issues such as:
    • BUG-2931 ‘run_time_permissions no longer triggers in attachments after requesting 0 permissions’
    • BUG-969 ‘teleporting breaks collision detection state for volumedetect objects’

Obviously, I’ll have more on the deployments for the week when details have been published by the Lab.

Server-Side Baking / Appearance Deployment

The enabling of SSB/A on LeTigre on Wednesday July 10th proceeded smoothly, and there have been few problems noted. Most testing the capability are reporting it functions smoothly and are seeing improved avatar skin / clothing layer rendering with few periods of extended avatar blurring.

It’s not entirely clear how SSA/B will proceed from here; it appears as if the Lab plans to keep to the cautious approach previously indicated by Nyx Linden rather than rushing forward to enable the capability right across the grid. As such, Log Linden indicated at the Server Beta meeting on Thursday July 11th that either BlueSteel or possibly – given it has a slightly larger number of regions than BlueSteel – Magnum will have SSB/A enabled next.

Have you updated to a viewer supporting SSB/A?
Have you updated to a viewer supporting SSB/A?

Log Linden came in for a number of additional questions on the new baking / appearance service during the Server Beta meeting on Thursday July 11th. Although I’ve covered the topics asked about in previous reports, I’ll note them here for ease-of-reference:

  • The baking / appearance service is entirely separate from the simulator servers, operating on its own dedicated servers. This means that when the service caches your outfit, it is effectively cached “grid wide” and not the simulator you happen to be on when you update your outfit
  • When two avatars are wearing exactly the same outfit, they both use the same copy of the outfit cached on the baking / appearance service. The Lab is aware this will only happen on rare occasions, and will most likely only happen in the case of library avatars
  • How long an outfit remains cached on the baking / appearance service depends on how popular it is. So if it is an outfit you are frequently changing into, it is liable to remain cached for longer than an outfit you wear, change out of and then leave it unworn for several days or more
  • Outfits which are removed from the baking / appearance cache are not deleted from inventory or anything – it just means that the baking / appearance service no longer has the data for the outfit, and so must wait until the outfit is uploaded from the viewer once more (although this shouldn’t visibly slow down the baking process)
  • Rebaking functions differently on SSB/A enabled regions: rather than uploading your appearance from your viewer to the server for distribution to the users around you, using the rebake (CTRL-ALT-R or menu option) causes the viewer to request a fresh download of your cached appearance
  • Should the server cached version of your appearance somehow become corrupted, you must add a clothing item in order to force a fresh update of your appearance from your Current Outfit Folder to the baking / appearance service
  • When you enter an SSB/A-enabled region for the first time, your own avatar will initially render grey before your skin and clothing layers re-render correctly. This is expected behaviour and due to you switching-over to the new baking / appearance service.

Anomalies

There have been some reports of odd anomalies in LeTigre regions (where SSB/A is enabled). These include:

  • A report that if you wear a 50% gray textured skin, you appear invisible to SSA viewers
  • Rather than appearing as a cloud in SSB/A-enabled viewers, avatars on non-SSB/A viewer entering LeTigre regions will actually render, but any outfit changes they make won’t render to those using SSB/A-enabled viewers unless they teleport to a non-SSB/A enabled region to change, and then return to LeTigre (something which will obviously not be possible once SSB/A is grid-wide).

Notecard Issue

One problem which has been confirmed with LeTigre regions is BUG-3203 (Error when adding contents to a notecard and saving). If you create a notecard on a LeTigre region and embed another item within it (another notecard, an LM, a texture), the notecard will not save, but will generate the following error: Unable to upload (asset ID number) due to the following reason: The server is experiencing unexpected difficulties. Please try again later.

General Feedback

Overall, feedback appears to be positive, going on comments in the forum deployment thread and made at the Server Beta UG meeting. Some people have reported performance dips while on LeTigre regions and attributed them to SSB/A, although this may be a placebo effect. There have also been so reports of viewer crashes when trying to walk between a non-Le Tigre region and a LeTigre region.

Most people who have carried out comparative testing between LeTigre regions and regions on other channels are reporting that avatar rendering on Le Tigre is significantly faster, with avatar skins and clothing appearing fully rendered in around 15 seconds in regions with a reasonable number of other avatars, and none of the general fuzziness or “double rebakes” which can be witnessed on the “old” baking process. Some are also reporting that they are seeing some improvements on non-SSB/A regions when using an SSB/A-enabled viewer, although this again may be something of a placebo effect.

My own rather quick-fire tests with the SL viewer 3.6.1.278007, Firestorm 4.4.2 and Exodus 13.7.9.1 beta revealed no discernible issues. The majority of avatars tended to render together with visiting regions with a reasonable number on them. I didn’t notice any clouds around me in the more crowded locations I dropped into and where I did experience performance drops, although I put those down to the fact the locations (a club and a hub) were busy with 15-20 avatars in each and had a lot going on rather than being anything intrinsic to SSB/A, nor did I witness anyone remaining grey for an inordinate amount of time.

Related Links