SL projects update week 18 (2): server releases

Deployments for Week 18

The week 18 deployments make for interesting reading.

SLS Main Channel

On Tuesday April 30th, the Main channel was rolled back to release 13.04.05.273580, as a result of a widespread performance issue.

Release Candidate Channels

  • BlueSteel and LeTigre: both of these channels should remain on the Experience Keys project, but will also be reverting some changes, due to the same performance issue which is affecting the Main channel – release notes
  • Magnum: should remain on the same server maintenance project as week 17.  This project brings some new minor features to LSL, and fixes some crash modes.  This update fixes the grid performance issue, and fixes an issue in which llDialog() messages sent to the object owner could be incorrectly throttled – release notes.

The performance issue which caused the Main channel to see the re-deployment of an older release was described by Simon Linden as being related to problems with regions locating their neighbours. “The performance problem was really showing up between any one region trying to locate another on the grid … the system was actually working, but too close to the cliff for comfort.” The re-deployment means that the new LSL AO capabilities can no longer be compiled / run on any Main, BlueSteel or LeTigre regions, until the supporting code is rolled-out once more.

There is still no further detail on the Experience Keys project and whether this may / may not be more than a deployment of the Advanced Creation Tools permission system.

Interest List Update

Andrew Linden has been working on fixing a bug related to Meeroos (but which I’ve seen affecting other animals as well), which he describes as, “If you turn your camera away from a crowd of Meeroos, wait several seconds, then turn back around… the Meeroos will be updated, but not quite in the right order. So sometimes you’ll see a head move to the new position, then a fraction of a second later the rest of the body.  So I have a theoretical fix that doesn’t crash the simulator (anymore).” The fix in question has yet to be tested and QA’d, so there is no time frame for any release.

“Missing” Prims

While talking about the interest list work, Andrew answered a question on missing prims / linksets, again acknowledging it to be a viewer-side issue, before going on to say:

We think maybe it is fixed in a new viewer. But this new viewer I mention happens to be very crashy, so we haven’t opened up the source code for it yet nor have we submitted it to our QA team since they’ll just crash …  This is the viewer that goes with our new interest list changes which I mentioned a few weeks ago and people were wondering when the code would be put up on a public repo.

"Missing" prims - viewer-side fix possibly on the way?
“Missing” prims – viewer-side fix possibly on the way?

So a viewer-side fix, along with viewer-side interest list updates, looks to be somewhere on the horizon.

Region Crossings

There have been a number of reports of region crossings worsening again after seeing a significant improvement with the release of the fix for BUG-1814. A common issue has been avatars becoming “snagged” at a region boundary while the vehicle they were travelling in continuing on its way, sometimes being returned to their Lost and Found folder from a location two or three regions beyond where the avatar became stuck. Both prim/sculpt and mesh vehicles are affected when the problem occurs, and it is an issue which had been encountered prior to the widespread deployment of the BUG-1814 fix.

Getting "snagged" at a region crossing while my aircraft flew on was a problem I encountered several times over Blake Sea early in April. The problem has again manifested itself to many, and I've again encountered it while flying my mesh Spitfire Mark IX
Getting “snagged” at a region crossing while my aircraft flew on was a problem I encountered several times over Blake Sea early in April. The problem has again manifested itself to many, and I’ve again encountered it while flying my mesh Spitfire Mark IX

I’d actually encountered the problem on April 4th during a series of region crossing tests, but the problem no longer appeared to be occurring by the middle of the month.

The issue of region crossings was raised at the Simulator Meeting on Tuesday May 30th, but the discussion was dominated by the problems being encountered by one particular type of train. In commenting more generally on region crossings, Andrew Linden said, “I agree that region crossing on vehicles needs more work. I can’t promise that I’ll be working on that as soon as I’m done with this interest list project, but I’ll try to bring it up in the next ‘what do we work on next’ brainstorm that we have.”

SL projects update week 18 (1): viewer, SSB/A and materials

Server-side Baking / Appearance

The viewer-side SSB/A code continued in the SL beta viewer with a release on April 24th (3.5.1.274588). However, the crash rates from that version were sufficiently low for the go-ahead to be given for the deployment of the SL viewer with the SSB/A code incorporated, which reached public release on April 30th (3.5.1.274821).

SSB/A in the SL release viewer: I'm running the SSB/A-enabled SL release viewer and can see both myself and my alt (running SSB/A-enabled Firestorm) rendered correctly, and she can see fully rendered.
SSB/A in the SL release viewer: I’m running the SSB/A-enabled SL release viewer and can see both myself and my alt (running SSB/A-enabled Firestorm) rendered correctly, and she can see fully rendered.

This now means that the following viewers and clients are, at the time of writing, now SSB/A capable:

  • SL viewer 3.5.1.274821 (release)
  • Firestorm 4.4.0.33270 (release)
  • Kokua 3.5.2.27969 (development)
  • Nirans 2.2.0.2692 (alpha test)
  • Cool VL 1.26.8.2 (stable release)
  • Kirstens S19.1.19.4 (unsupported)
  • Singularity 1.8.0.4114 (release)
  • Lumiya 2.4.4 (release)
  • Metabolt version 0.9.66.0 (Beta)
  • Radegast 2.12 (release)

The remaining major players for Second Life – Catznip, Dolphin and Exodus will doubtless have SSB/A versions out in the very near future.

Z-offset

Cinder Roxley is working on an alternate approach to the issue of the z-offset and is meeting with some success. However, there is still a problem with the distance offset being inconsistently reported between the user’s viewer and by other viewers (so the avatar may appear at a different height above ground / an object when seen by others in comparison to how they see their own height. Cinder is continuing to work on the problem. If she is successful, the code will doubtless be made available to all TPVs should they wish to adopt it.

Current Outfit Folder Corruption

I reported on this issue in week 17, wherein a Current Outfit Folder (COF) corruption can leave a user unable to log-in to SL and – unless they have a Premium account – beyond official help. This has been something of a longstanding issue, as per JIRA SVC-7653, and has an associated work-around. The concern here is that the workaround will no longer work once SSB/A is enabled server-side.

Commenting on the situation, Nyx Linden said, “we’re looking into a number of fixes around COF for followup releases,” before going on, “Yep, we have people looking at this. Please do continue to let us know as you see cases come up. I’ll sync up with our engineers looking at this and make sure that we have these cases covered.”

SUN-69

Whirly Fizzle recently reported an issue arising from the recent SSB/A code – removing any worn item from avatar results in all temp attachments being taken off (see JIRA SUN-69). This problem occurs whether or not an avatar is on SSB/A regions. It was first noted in the SL development and beta viewers, but appears common to all viewers with the SSB/A code, including the new release version of the viewer referred to above (3.5.1.274821).

Server-side Deployment

With the arrival of the viewer SSB/A code into the SL release viewer, deployment of the server-side code is liable to commence on the main grid. As previously noted in these updates, the plan is for a “constrained” number of regions on Agni to be SSB/A-enabled in order to load test the system. It appears likely that these regions will not be a part of the normal Release Candidate channels (although this is not absolutely clear).

The purpose of this action will be to further stress / load test the new server-side baking service and (hopefully) ensure there is sufficient hardware deployed to support the capability and that there are no unexpected issues arising from large numbers of people starting to the use the system.

Continue reading “SL projects update week 18 (1): viewer, SSB/A and materials”

SL projects report week 17 (3): server, group bans and Oculus Rift

Server Deployments – Week 17

The scheduled server channel deployments took place as planned this week.

The SLS Main channel

As previously reported, this received the update package deployed to the three Release Candidate channels in week 16, primarily comprising the new server-side LSL Animation Override capabilities, complete with a fix for BUG 2164release notes

BlueSteel and LeTigre

Both of these channels received the first part of a new experience tools project – referred to as the “Experience Keys project” in the release notes.

Interestingly, the release notes refer to BlueSteel and LeTigre receiving server release 13.04.19.274370; however, the viewer reports both of the channels running 13.04.05.273550. I contacted Maestro on this, who replied, “There was an error during the roll, so a slightly older version (which doesn’t include the changes from this week’s main channel update) was deployed.”

Not too much is known about the new Experience Keys project, although the emphasis on “new” indicates this is more than just a deployment of the outstanding permissions system for the current advanced creation / experience tools, and speculation has been running high. At the Simulator User Group meeting on Tuesday 23rd April, Simon Linden indicated he was unsure as to what he could / could not say on the matter (particularly as it appeared the documentation was still being written-up).

Commenting on the Bluesteel / LeTigre deployment at the Server Beta meeting on Thursday 25th April, Maestro Linden was also somewhat circumspect on the matter, commenting, “The team doesn’t want to announce the features yet, so I can’t give many details … some other parts need to be released for the new features to be usable. So ideally, nothing visible should change there.”

The reference to “some other parts” which need to be released for these new features may include a viewer update. Whether such an update will appear ahead of or behind the materials capabilities (still currently in a project viewer) is unclear at this point in time.

Magnum

Magnum received additional LSL support for new HTTP contents types, as document in the release notes. It also received a change to how certain message types are handled by the server, which Maestro described thus:

There’s a change to make the server smarter about how it throttles certain messaging types to prevent certain types of ‘DoS’ attacks, where a ‘bad’ object could prevent your avatar from getting llDialog notifications from other objects. All objects owned by UserA share the same throttle for sending llDialog() messages to UserB, but objects owned by anybody else would have a separate throttle pool.

This should hopefully reduce the incidences of iiDialog being used in spamming attacks which can result in the viewer either being severely slowed down or crashing altogether.

SL Viewer

The beta viewer gained a further release (3.5.1.274558) which reached public availability on April 24th, containing further CHUI and SSB/A dixes and updates, as detailed in the release notes. The development viewer also received a further release (3.5.2.27469) which also gained public availability on April 24th.

Group Bans

Baker Linden: Group Ban work coming, just not quite yet
Baker Linden: Group Ban work coming, just not quite yet

Baker Linden has started working on an update to the code for managing groups which will allow group owners / moderators to ban users who create problems (e.g. those who spam groups, people who are persistently abusive in group chat, etc.). This work is in response to JIRA VWR-29337. In my last report on this, Baker has written-up the documentation for the work and was having some other Lindens cast an eye over it.

Attending the Server Beta meeting on Thursday April 25th, Baker provided an update on his activities. “I’m still working on group bans, but I decided to fix a couple small bugs first. They both relate to searching people using the choose resident floater. They’re in the system where I’ll be adding group ban stuff, and now that I can test the changes, I can get them pushed to an RC. However, he did go on to say, “It’s going to be a while before Group ban stuff is ready.”

Second Life and Oculus Rift

There has been considerable interest in the Oculus Rift headset and its potential for use within Second Life, as reported back in week 14 and more recently. Jon Brouchoud in particular blogged on why SL would be a “killer app” for the headset, and a video I featured back in week 16 has also appeared on numerous SL blogs (hardly surprising, given it has pretty much gone viral where the Oculus Rift is concerned 🙂 ).

On Wednesday April 24th, Hamlet Au covered the fact that the Lab is already looking to integrate the headset into Second Life, and have given official confirmation, with company spokesman Peter Gray (Pete Linden) quoted as saying, “We plan to strongly support Oculus Rift. That means code, client, and server-side, to make the Oculus Rift experience excellent in Second Life.”

Continue reading “SL projects report week 17 (3): server, group bans and Oculus Rift”

SL project reports week 17 (2): COF corruption and SSB/A

At the Open-source Dev meeting on Wednesday April 25th, Whirly Fizzle raised a concern on the status of JIRA SVC-7653, which relates to Current Outfit Folder (COF) corruptions leading to an avatar being unable to log-in to Second Life which may have an impact on the forthcoming Server-side Baking / Appearance switch-over.

The problem was first publicly reported by AngusGraham Ceawlin in February 2012, and at the time became of the focus of a campaign to Free Angus. After a total of some 63 days unable to log-in and with the (particular) support of Whirly herself and Nicky Dasmijn, Paspund Resident and Izzy Linden, Agnus was indeed freed. However the problem has become a matter of concern again because SVC-7653 remains marked as “unresolved”, and a workaround developed for it looks set to be “broken” when SSB/A goes live.

The "Free Angus campaign poster" produced by Sparkles Alchemi and used in a number of blog posts in early 2012 to highlight Angus' situation
The “Free Angus campaign poster” produced by Sparkles Alchemi and used in a number of blog posts in early 2012 to highlight Angus’ situation

The COF Corruption Issue

When this particular issue occurs, the user has no clear indication that there has been an inventory corruption specific to the Current Outfit Folder. There are only tell-tale clues, which Whirly Fizzle documented (on behalf of Kitty Barnett) as being:

  • Only one account is affected
  • A crash or timeout disconnect will occur at log-in, usually around “Downloading clothing…”
  • The affected account will crash/disconnect on any V3-based viewer or a viewer that uses COF
  • A clean install, cache clear, replacing outfit on a non-COF viewer will not fix the problem
  • The associated viewer logs will usually show:

INFO: newview/llappearancemgr.cpp(1998):LLAppearanceMgr::updateAppearanceFromCOF : starting

Generally (but not always) followed by numerous warnings:

WARNING: newview/llappearancemgr.cpp(2891) : WearablesOrderComparator::operator(): Warning # 0: either item1 or item2 is NULL

Followed by a crash or a timeout.

The Workaround

While LL’s support staff do have scripts to deal with various inventory corruptions and issues, under current rules, they will only run these scripts for Premium accounts. While it may be possible to get further assistance from the Lab if you’re not a Premium account holder, it is certainly going to take a good deal longer to gain assistance. As a result, Kitty Barnett developed a workaround for non-Premium members, which Whirly included in her comments on SVC-7653:

  • Use a viewer which does not have COF support and log-in to SL. Currently, Imprudence is possibly the most easily obtainable such viewer
  • Replace outfit with a Library avatar. Thus must be a complete outfit change (every layer / attachment)
  • Log out of the viewer
  • Launch a TPV which uses the VerifyInitialWearables debug setting (such as Catznip, Exodus, or Firestorm – note that the official viewer does not have this setting)
  • At the log-in splash screen:
    • Go to the top menu bar and select Me/Viewer > Preferences > Advanced and tick Show Advanced Menu
    • From the top menu bar, go to Debug > Debug settings > VerifyInitialWearables and set it to TRUE
  • Login to SL.

This should result in a message being sent to the server by the viewer using the VerifyInitialWearables which will result in the COF corruption being “fixed” and the avatar being able to log-in successfully to SL. A change of outfit should then verify that all is well.

The Workaround and SSB

Tests carried out by TPV developers have shown that the VerifyInitialWearables message sent by the viewer is ignored by the SSB/A servers. This has resulted in concern that unless the Lab have developed a resolution for this issue,  anyone encountering it once SSB/A is “live” will not be able to utilise the documented workaround, and they’ll effectively be unable to log-in to Second Life unless they are a Premium member (or upgrade).

The matter has apparently been brought to the attention of the SSB/A team (led by Nyx Linden), who have been engaged in a wide range of inventory updates and associated work, “More than they hoped would be needed,” as Oz linden put it at the Open-source Dev meeting. But whether or not their work extends to fixing this particular problem is unclear. SVC-7653 remains “unresolved” but inactive – so it is perhaps possible a fix has been developed internally by the Lab without SVC-7653 being updated. However, until greater clarification is given, this is likely to be a subject of note at User Group meetings which deal with SSB/A matters.

Related Links

SL projects update week 17 (1): Server releases, SSB/A, particles

Server Deployments – Week 17

SLS Main Channel

On Tuesday 23rd April, the SLS Main channel received the update package deployed to the three Release Candidate channels in week 16, primarily comprising the new server-side LSL Animation Override capabilities, complete with a fix for BUG 2164, wherein the new capabilities could conflict with built-in animation poses in chairs, etc., as discussed in my week 15 updates.  This deployment additionally includes the slight region performance improvement when there are no pathfinding characters present. Release notes are available.

Release Candidate Channels

On Wednesday 24th April, the RC channels should receive the following updates:

  • BlueSteel and LeTigre: should gain a new project which brings preliminary server-side support for experience permissions – release notes
  • Magnum: should gain a new server maintenance project.  This update brings some new minor features to LSL, and fixes some crash modes – release notes

So the long-awaited experience tools / advanced content creation tools permissioning system looks to be finally on its way, a little under a year since an exploit caused them to be withdrawn after their initial deployment.

Viewer Updates

Development Viewer Updates

The SL development viewer has been through a rapid series of updates over the last few days of week 16 and the start of week 17, with version 3.5.2.2744 released on Monday April 22nd. While release notes are not available, it is thought it includes (among other things), the fix for music stuttering every few seconds when using FMOD EX, as submitted to LL by Latif Khalifa  (see OPEN-173).

Particle Project Viewer Soon (?)

It appears a new particle project viewer may be on the horizon in the near future. If it does appear, it is likely to be primarily aimed at testing a new capability to help deal with griefing attacks which use particle emitters. Speaking at the Simulator User Group meeting, Simon Linden said:

We’re doing some testing and may get a project viewer out, which would allow you to test it and (I believe) let other viewers check out the source code. This is right-click on a particle, and it kills the generator … We’ve tossed around a few ideas about blocking particles in other ways but definitely want to get these first steps out the door.

This sparked further discussion on ways and means to help stop particle griefing, which has been on the increase across the Mainland of late. However, given that particles are a viewer-side effect, “stopping” them is actually easier said than done from a server-side perspective. One idea which gained some interest, if it is feasible, would be a block of any particle emitters belonging to an avatar banned from a region / parcel.

In the meantime there are means of stopping the viewer from rendering all particles – such as CTRL – Alt – Shift – +, or going to Preferences > Graphics > and dropping the Particle Count slider to zero. These again only work in your own world view, and while not always ideal, do present an option.

Asked if such a particle project viewer might include the new LSL particle capabilities already deployed server-side, Simon could only say, “Hopefully yes … I’ve been working on getting that chunk of code solid for the last few days. Unfortunately it’s pretty easy to crash at this point and so it’s not ready for consumption.”

The particle ribbon effect  is a particle which repeats the texture horizontally, and can follow the object creating them, which offers a lot of potential uses, as which forms a part of the new particle capabilities – soon to be seen within a project viewer? – Time will tell.

Server-side Baking / Appearance

The viewer-side code for SSB/A is starting to appear in more TPVs:

  • Cool Viewer has had SSB/A support in the “Experimental” branch for a while, and this moved to the “Stable” and “Legacy” versions on April 20th – see the Cool VL website and release notes
  • Singularity released version 1.8.0 with SSB/A support on April 21st, which I’ve reviewed
  • Firestorm released version 4.4.0 with SSB/A support on April 22nd, which I’ve reviewed
  • Kukua has had an experimental / development version since the start of April (version 3.5.1) which also incorporates SSB/A.

The status of the SL beta viewer with the SSB/A code will take place around the middle of week 17, when a decision taken as to whether to incorporate the code into the SL release viewer or run a further beta release. The expectation is that the code will move to the release channel unless a last-minute significant issue is found within the code which prompts a further run in beta.

Assuming the code does make it to the release channel, it is likely that a release viewer will appear early in week 18 rather than late week 17, due to the time required to test and QA a build.

SUN-72 – Fix Submitted

One issue which is known to exist within the current beta release of the SSB/A viewer code and which has caused considerable problems is SUN-72 – if you have inventory items with special characters (including the likes of asterisks) in their names, they will fail to load. Similarly the use of accented characters (e.g. such as René) in things like chat log path names, the logs are not saved.

Nicky Dasmijn of the Firestorm team developed a patch for this specific problem which is already incorporated into the recent Firestorm 4.4.0 release, and which has been contributed to, and accepted by, the Lab for integration into the SSB viewer code. Hopefully, it should be appearing in the next SSB viewer release, whether beta of SL release viewer.

Sun-38 – Avatar / z-offset

As noted in my last major update on SSB/A, the ability to offset an avatar’s height on-the-fly to accommodate various animations (e.g. kneeling), or to adjust an avatar’s height when sitting / walking of floors, etc., and which will effectively be “broken” as reported in  SUN-38, remains an issue for many.

While the Lab have produced an alternative approach which addresses some issues around avatar height offset using new appearance sliders, and which Nyx Linden and others have been continuing to tweak, there is no solution on the horizon for maintaining any form of on-the-fly adjustment; nor does the proposed solution work with no-mod shapes, as noted when I first  reported on the “fix”.

During the Content Creation User Group meeting on Monday, April 22nd, a request was made for the Lab to not shut-down the existing baking service until such time as  SUN-28 is “solved”, which prompted Nyx to comment:

SUN-38 is not currently considered a blocking issue for our initial release. It’s on our list to investigate, but we don’t have a patch or update immediately. There is a new hover parameter, which should work for attachments that affect your avatar, but the bug reported is also discussing the need to adjust the offset on the fly without changing wearables. Having both the new and the old system enabled is not an option … We are investigating options, but it is not a hard requirement for the initial release.

Continue reading “SL projects update week 17 (1): Server releases, SSB/A, particles”

SL projects update: week 16 (4): More on SSB, materials and FMOD Ex

Server-side Baking / Appearance

Viewer Releases

An updated beta viewer was released on April 19th (3.5.1.274264, with release notes here), which contains a range of updates, including several for SSB/A. Speaking at the TPV Developer meeting on Friday, 19th April, Oz Linden indicated that the current plan from the Lab is that this is likely to be the last beta viewer release for the SSB/A code unless a major blocker shows up. Assuming this doesn’t happen, then it is more than likely the SSB/A will move to the SL viewer release channel and arrive as a viewer update towards the middle of week 17 (week commencing Monday, 22nd April), which should hopefully be an automated update for most users on the SL viewer, given the majority appear to keep that option active on their viewers.

Given us, it is liable that we’ll start seeing more TPVs updates appearing in the near future – and people using TPVs are going to need to start installing and using those updates rather than remaining with older versions of their viewer if they are to avoid the “grey people” syndrome as the server-side of SSB/A is deployed.

Users will need to update to version of their preferred viewer supporting SSB/A as they are made available to avoid seeing grey people
Users will need to update to a version of their preferred viewer supporting SSB/A as they are made available if they are to avoid seeing grey people once the deployment of the server-side of SSB/A commences

Server-side Deployment

The precise means by which the server-side will by deployed is still not absolutely clear. As noted a number of times in this blog, Nyx Linden is hoping that things will progress somewhat cautiously, possibly starting with a set of “carefully selected and constrained regions on Agni” as the viewer code reaches the SL release viewer. This still appears to be the case, but it is possible that it will not – as Nyx has previously hinted – roll through the Release Candidate channels.

“I don’t know whether it will go through the normal RC process,” Oz Linden commented at the TPV Developer meeting, “Because it’s not actually a server software change; it’s a configuration change, so they don’t need to deploy it through the RC progression. All they have to say is, ‘Yes, throw the switch!'”

If this approach is taken, it’s currently not clear whether or not it will require a region restart.

In terms of time frames as to when this might happen, things are similarly unclear at this point – a lot depends on how well the testing on the selected Agni regions progresses. However, Oz suggested that the time frame in which the “switch may be thrown” to be anything between 2 and 6-8 weeks from when the viewer-side code appears. His analogy was that of a bell curve, wherein the switch-over could occur at the top of the curve – but could, if circumstances dictate, occur at either end of the curve.

Communications

It is still hoped that there will be a concerted effort on the Lab’s part to communicate server-side baking ahead of any server-side flipping of switches. A blog post is apparently in preparation, which should go out when the SSB/A code is issued in the release viewer; whether this will be supported by other means of communicating the changes is unclear.

However, communications on the whole is not easy – even with the TPVs also liable to be spreading the word through blogs, etc., (as Firestorm have already). This is because even with the official blog, TPV blogs and blogs such as this one, the vast majority of SL users do not read blogs.

Matters are also further complicated by the fact that there are over 1700 different viewer strings which are used to connect to SL (even if not on a daily basis). These not only include the official viewer and current versions of TPV-registered viewers, but also many instances of older versions of TPVs, Snowglobe (1.x) based viewers, several versions of the official 1.x viewer (some of which date back over 5 years), viewers which are not registered with the TPV directory, self-compiled versions of various viewers, and so on.

As such, whatever the effort made to communicate the arrival of SSB/A is liable to be missed by a good number of users and people are going to find themselves facing grey avatars as a result of the switch to SSB/A, because many of these additional viewer strings will not have the necessary viewer-side code. This inevitably means that there is going to be some disruption and upset. While this in turn doesn’t mean that attempts to communicate the coming change shouldn’t be made – but it does mean that even with the best efforts of the Lab and TPVs combined in communicating SSB/A, there is going to be an outcry. So anything all those who are aware of the upcoming changes can do to communicate it to others – particularly when there is a visible log post from LL on the matter which can be referenced – can only help lessen the volume of that outcry.

Continue reading “SL projects update: week 16 (4): More on SSB, materials and FMOD Ex”