SL projects updates 20/1: server, viewer, LSL and materials

Server Deployments, Week 20

There was no Main channel deployment or rolling restart on Tuesday May 13th, and neither the BlueSteel or the LeTigre RC channels will receive an update or should undergo a restart on Wednesday May 14th.

The Magnum RC should receive a new sever maintenance project on Wednesday May 14th, which includes a bug fix for a networking-related issue that sometimes affects busy sims.

SL Viewer Updates

The SL Maintenance viewer was updated on Monday May 12th to version 3.7.8.289922. This viewer includes multiple fixes to Mac viewer; fixes in Recent tab, Chat, LSL editor, land management, etc; GPU table updates; crash fixes & performance improvements.

LSL Functions for Materials

The subject of scripted control for materials was once again raised at the Simulator User Group meeting on Tuesday May 13th. Commenting on the matter, Simon Linden said:

I am looking at it but not promising anything. We’re trying to be really careful to understand how the server and viewers will react when stressed with a lot of material churn. From what I can tell, fast-moving material-based animation will not work well … that’s likely to be throttled or blocked somehow. But supporting something like a hud or other control that could adjust the look of an object … where it’s done rarely … is definitely possible.

As noted the last time this subject was raised, there are concerns over how LSL control of materials might impact system performance, either deliberately (via rapid and multiple flipping of maps, hence Simon’s comment on throttling the speed at which changes could be made), or unintentionally, such as using them with objects which may already have a large performance impact (such as animated mesh tails).

During the meeting, there was discussion on options for animating normal and diffuse maps, remembering that they can already be animated in lockstep with their attendant texture (diffuse) map. During this discussion, Simon commented on some of the difficulties in animating  materials independently of the texture map:

The materials LSL support would include changing the offset, repeat and rotation values for the two maps, just like for regular textures. The update problem hits if you look at the way materials have been optimised between the server and viewer and how updates are sent. Materials are referred to by a number ID … so you get updates that say “this face has material 1234” on it, the viewer, if it doesn’t know what 1234 is, has to ask the server.

Now, if you change the offset … you have a new material 34356, the viewer has to again find out what that is, but this time it already has the actual specular and normal maps, so no download there.  And when you switch back to 1234, it has all the info and can draw it faster.

Summing-up the situation in general, Simon concluded, “I hope there will be something to play with eventually on the beta grid … we’ll probably want to experiment there and figure out what kind of limits are effective.”

SL projects updates: TPV developer meeting, Friday May 9th 2014

A TPV developer meeting took place on Friday May 9th. The core items discussed in the meeting are reported below, with timestamps in the relevant paragraphs indicating the point at they are discussed in the video embedded here. My thanks as always to North for the latter.

SL Viewer Status

[00:35] As noted in part one of this week’s updates report, the de facto viewer was updated on Tuesday May 5th with the promotion of the Interest List viewer (version 3.7.7.289461). The viewer contains what’s being referred to as a number of “non-trivial” merges, as the team responsible for the viewer took the opportunity to clean-up “a lot of old and unpleasant code”, and this “touched on a lot of things”. As such, it may be a while before this code filters into all TPVs.

[01:27] On Thursday May 8th, the SL Share-2 RC updated to version 3.7.8.289775 (download and release notes). This viewer includes the ability to upload Tweets and snapshots to Twitter and / or snapshots to Flickr, and to use pre-set filters on images being uploaded to either service and / or to Facebook, and to create your own filters.

The remaining viewers in the release channel (Sunshine / AIS and the Maintenance viewer) and the Zipper project viewer will be updated in week 20.

[02:11] There are two or three other viewers which are expected to be appearing in the near future. One of these contains a number of Snowstorm contributions (such as STORM-1831, currently awaiting two minor bug fixes), Baker Linden’s group ban work should be generating a viewer soon (see below for more), and there is a viewer which contains a series of memory leak fixes which is currently in QA.

Group Ban List

[03:40] Baker reports he has two “major” bugs and three or four “minor” bugs still to deal with; however, it doesn’t appear is if these are going to stop the viewer arriving as a project viewer. As noted in part two of this week’s report, the repository for the code has already been made public, and TPVs have been invited to pull code from the repository if they’re in a position to do so (the group ban viewer is built to LL’s 3.7.8 code base).

Obviously, and again as noted in part two of this report, the code will not be usable on the main grid until such time as the server-side changes have been deployed, and this isn’t likely to happen for a couple of weeks or so, so don’t expect it to be appearing in release versions of any viewers for a while.

The server-side code is available on a channel on Aditi (DRTSIM-234 14.05.05.289712 – which includes the Morris region where the Server Beta meeting is held and now includes the BUG-5929 fix), and there may be a grid-wide Aditi deployment of the server-side code. If this is the case, it will likely be confirmed via a Server Beta group meeting.

One aspect that has not been looked into as yet is ensuring that when someone is ejected / banned from a group, they are also ejected from group chat. Currently, due to the way the back-end services operate, if someone has the group chat window open when they are ejected from a group, they can continue to chat / spam into the group chat up until the point where they close the window. Commenting on this, and given that Simon Linden has been working on the chat service, Baker has indicated that he’ll look into things with Simon and see if this problem cannot be resolved.

Leap Motion Integration

[12:18] In November 2013, Leap Motion approached Linden Lab about integrating their gesture controller into the view.  Due to the amount of work the Lab had on its plate, the work was handed-off to TPV, with members of the Firestorm team working with Leap Motion to get things integrated.

Since that time, the work has been subject to a number of hiccups – including the need for Leap Motion to update their software. As it stands, the work is slightly stalled as the Firestorm team no longer have the resources needed for the work, so a call has gone out to TPV developers who are willing to take a lead in bringing this work to fruition.

Third-party Library Work (Webkit et al)

[26:14] Monty Linden is continuing his work in cleaning-up the third-party libraries used within the viewer build process. This work has been focused of late on Webkit, which is used for a number of tasks, such as powering the built-in web browser and to display profiles, and is used with Media on a Prim (MOAP) and many in-world televisions. However, Monty has more recently been working on the COLLADA DOM library as a means of “taking a break” from Webkit. He describes this as the “last big one” on his list.

Despite still having to finish-up with Webkit, Monty is already in a position of being able to use a windows version of the viewer which makes use of his updated and cleaned-up libraries, although he emphasises the work is not ready to enter prime-time use as yet.

Continue reading “SL projects updates: TPV developer meeting, Friday May 9th 2014”

SL projects updates 19/2: group bans, miscellaneous items

Server Deployments Week 19 – Recap

There were no server deployments!

Group Chat

As noted in part one of this report, the group chat updates were deployed to the back-end chat servers on Monday May 5th. The changes to group chat should be subtle, and may not be observable to many. Additional analytics are included in the code, which should provide further pointers on what else may need addressing going forward.

Group Ban Lists

Obligatory Baker Linden shot :)
Obligatory Baker Linden shot 🙂

Baker Linden’s work on adding the ability to ban troublemakers / spammers, etc., from groups with open enrollment is now getting relatively close to becoming available.

Baker has recently closed what is believed to be the last of the server-side issues, BUG-5929. This meant that if the name of the group owner was accidentally added to a list of people to be banned from a group, the ban process would fail, with no-one in the list either being added to the ban list or banned from the group (although other than the group owner, anyone selected for banning would be ejected from the group).

The expected behaviour would be for all those named (other than the group owner) to be added to the ban list, with those who were already members of the group also being ejected and banned. Baker’s fix is to ensure this is now the case, and it should be available shortly on Aditi for testing (channel DRTSIM-234 14.05.05.289712 – which includes the Morris region where the Server Beta meeting is held).

Viewer-wise, a project viewer with the new code is expected to appear very shortly (it was running through the build process during the Server Beta meeting on Thursday May 8th). This should be added to the Alternative Viewers wiki page when available. The repository for the code has now been made public, so TPVs can start looking at it – but again, given the status of the viewer as a project release, don’t expect the code to immediately start popping-up in TPVs.

HOWEVER, it may be a while before the new group ban functionality can be used on the main grid, as there is an initial back-end host code update required prior to anything being deployed to any simulator channel. According to Maestro Linden, the Lab will likely want to run those updates for a week to check for any unexpected regressions prior to putting any simulators on a group ban RC.

In the meantime, the group ban capabilities can be tested on Aditi either using the project viewer (when available) or the existing test viewer.

Other Items

“Welcome to the Hotel California” – BUG-5961

Trying to leave a group with a large membership list can prove problematic if the memebrship list takes time to load
Trying to leave a group with a large membership list can prove problematic if the membership list takes time to load

An old issue recently came to light once more with BUG-5961 (originally entitled “I cannot leave a group that I joined”, but with the description subsequently updated by Maestro to “Viewer attempts full fetch of member list before allowing user to leave group” in order to better reflect his findings following investigation).

It’s not actually clear if this is a one-off situation, or possibly more widespread, as the bug report is specific to the group “Akeyo”.

However, Maestro’s thinking is that the problem is linked to the download of the membership list, which even with the Group Services fixes introduced in late 2012, can still take time to complete with some larger groups.

Essentially, you cannot leave a group until the membership list has been loaded, as the viewer must check to ensure that when leaving, you’re not the last owner of the group. Should the membership list take time to download, this can lead to a temptation to click the Leave button again, causing the download to start-over, resulting in the list not loading, thus preventing you  from leaving it (hence the Hotel California quip, which I admit I stole from Maestro!).

The Lab is looking into this issue further, although it may be a while before any resolution is found. One workaround in the meantime is to run a client such as Radegast, which handles groups slightly differently to the viewer, and use that to leave the offending group.

Restore to Last Position

Restore to Last position was a popular feature which allowed anyone to take content to inventory and then re-rez it later at the same position. While there were issues with the capability (such as using it to rez an object in a different region, with a different topology to the one where it was originally taken back to inventory resutling in an object to “vanish”, as it rezzed underground or something), it was broadly seen as beneficial.

However, it was also subject to exploitation, which is why the server-side behaviour for it was changed by the Lab some time ago such that the function will only work if you have rezzing rights at 0,0,0 in a region. If you do not, any attempt to use Restore to Last Position will fail with a notification that you don’t have the required rezzing permissions. The viewer-side code for the capability was also removed from the SL viewer, although TPVs have retained it.

A further issue with the capability has been with No Copy objects. If Restore to Last Position is used on these when the user doesn’t have rezzing rights at 0.0.0 in a region, they not only fail to rez – they also vanish from inventory, requiring a relog in order to get them listed again.

However, BUG-5955 “Restore to Last Position (used only by TPVs) causes content loss” highlights a problem where at least one type of No Copy object can be permanently lost from inventory if Restore to Last Position is used even in a region where the user has rezzing permissions at 0,0,0. Not even a subsequent re-log sees the item reappear in inventory.

Given the unpredictable nature of Restore to Last Position, the Lab is considering removing or blocking all support for it viewer-side until such time as a fix for issues can be found / it can be made to work more predictably in all cases.

As an alternative, and given the function’s popularity, it has been suggested a restriction preventing its use on No Copy objects should be implemented. The Lab may be taking this under consideration. This is the option Firestorm have indicated that they intend to implement with their upcoming release (which may as a result be delayed until the code is implemented and tested).

SL projects updates 19/1: SL viewer, group chat and miscellaneous things

Server Deployments

There are no scheduled simulator deployments this week to either the Main or RC channels, and so no associated rolling restarted expected.

SL Viewer

The Interest List RC finally made it to the de facto release viewer with its promotion on Tuesday May 5th (version 3.7.7.289461). This leaves just three RC viewer in the release channel at present: SL Share 2 project viewer version 3.7.7.289497; Sunshine / AIS v3 RC  version 3.7.7.289441; and the Maintenance RC viewer version 3.7.7.289405. Please refer to my Current Viewer Release page for up-to-date information on all viewer releases.

 Group Chat

Simon Linden’s optimisation work for group chat was deployed across all of the back-end chat servers on Monday May 5th. while these should see some improvements in group chat (particularly in sending / receiving chat and moving between regions), Simon does warn that these optimisations are not expected to “fix” all of group chat. However, he will continue to work on further improvements as well.

Other Items

New Starter Avatars

Ebbe Altberg used one of the upcoming new starter avatars at the VWBPE conference in April (image: Strawberry Singh)
Ebbe Altberg used one of the upcoming new starter avatars at the VWBPE conference in April (image: Strawberry Singh)

During his appearance at the VWBPE conference in mid-April, Ebbe Altberg appeared using one of the new starter avatars. At the time he did, it was hinted that the new avatars would be appearing relatively imminently. However, almost a month on and they have yet to officially appear, although there is some speculation they’ll do so in May.

these new avatars are said to take advantage of some of the latest features in SL, which is being taken to mean that some / all are full or partial mesh. This has in turn raised questions as to whether it is wise giving new starters full mesh avatars, given they may not work with freebie items often offered to or picked-up by new starters.

LSL Functions for Materials

While there is no confirmation any work is being carried out on this (except, as Simon quipped, “perhaps in a parallel universe or something”), the Lab is still sounding out how and where such calls would likely be used, and the frequency with which such calls would be made.

The option of having scripted control of materials has been debated often, and still remains a desired item among builders and scripters. However, some of the concerns still remain – notably, have such capabilities might end up causing performance issues, deliberately or otherwise. Much has already been written on how rapid map flipping on multiple objects could deliberately impact performance and potentially result in viewer crashes, plus there are already animated mesh elements available which can also have a significant impact on viewer performance (some types of animated mesh tail can reportedly overload a viewer on a 32-bit system with out-of-memory errors in a matter of seconds), so there are also concerns that were this to be combined with the ability to change textures via script, they could (even unintentionally) have further dramatic impacts on performance.

One way around this would be to throttle the rate at which material maps can be changed via scripted command. What is interesting for the moment is that the Lab appears to have not completely closed the door on scripted control of materials, but is considering options and informally seeking feedback on potential use cases.

 

SL projects updates 18/2: group chat; group bans

Server Deployments Week 18- Recap

  • On Tuesday April 29th, the Main channel received the server maintenance project that was on the Magnum RC in week 17, comprising a fix for BUG-5533 and a crash mode fix.
  • On Wednesday April 30th, the Magnum RC had the server-side Sunshine  / AIS v3 code re-enabled (this code requires the use of the Sunshine RC viewer), and all three RCs were updated with the bug fixes deployed to the Main channel.

SL Viewer

There have been no updates to the RC viewers in the release channel during week 18  and no further releases, either RC or project viewers, so the SL viewer releases remain as per the last update to my Current Viewer Releases page.

Group Chat Optimisation

Simon Linden dancing at a Server Beta User Group meeting
Simon Linden dancing at a Server Beta User Group meeting

The code Simon Linden has been working on to improve group chat was deployed to a single group chat server, where it has been running for all groups starting with group_id “b”. Commenting on the work at the Server Beta meeting on Thursday May 1st, Maestro Linden said:

Simon’s been looking at the performance of that group chat server, and it seems to be running fine. So there are plans to update the rest of the group chat servers to the new version early next week. We won’t go so far as to say that group chat has been totally fixed, though – Simon has identified some other changes which could improve performance further.

A recent fix was made to IM sessions to correct the issue where it is possible to see “typing…” in an IM window when the other person isn’t actually typing (see STORM-1975), and questions were raised on whether this fix might be adding a load to group chat sessions, as the viewer-side code appeared to send the message during group chat sessions as well as person-to-person IMs. However, both Simon and Maestro Linden indicated that the notifications are simply ignored by the chat servers during a group chat session, so no additional load is created, although Maestro admitted it would be nice if the viewer didn’t send meaningless messages.

Aside from the back-end load, the biggest issue which occurs in group chat is when someone using it changes regions. When this happens, the chat service has to figure out where you are.

“The region you are on, your viewer and the back-end database all know where you are, and keep updated very fast. The chat servers, however, aren’t kept in perfect sync because that would be very hard to do with 50000+ people moving around who are all in 42 groups,” Simon said, in explaining the problem. He added, “It [the chat service] doesn’t track missing messages … but if it can’t send one to your agent, it then has to ask where you really are and then it sends there,” all of which takes time, delaying the receipt of group chat messages.

In describing the changes made, Simon concluded:

The new code we have out now is a bit more efficient, but more importantly it has more metrics and it showed me that the performance problems are in a few other areas … it turns out the updates to keep the list of people in the group chat updated are really significant. It gets worse, of course, in large groups – more people coming and going, and more people who need the updates.

Group Ban Lists

It appears a server-side deployment of the code require to manage the new group ban list functionality is drawing close. There is still work to be done viewer-side, but recent testing on Aditi resulted in a number of JIRAs being filed, and the associated server bugs have been stomped on by Baker Linden, with help from the likes of Caleb Linden.

SL projects updates 18/1: miscellaneous items

Server Deployments Week 18

As always, please refer to the server deployment thread for the latest updates and news.

Main (SLS) Channel

On Tuesday April 29th, the Main channel received the server maintenance project that was on the Magnum RC in week 17, comprising:

Release Candidate Channels

On Wednesday April 30th, all three RCs should receive the same maintenance package. This comprises the same fixes as deployed to the Main channel and sees the Magnum RC rejoin BlueSteel and LeTigre with AIS v3 support, which requires the use of the Sunshine RC viewer.

SL Viewer

As per my recent post, the Lab have updated their SL system requirements page for Windows and Mac OSX to better reflect the public status of both operating systems.  There is an official blog post on the subject as well, which essentially confirms that Windows XP and OSX 10.6 are considered as no longer supported. While accessing SL from computers using these OS versions will not be blocked, users will no longer be able to obtain SL-related support should they encounter issues, and the Lab is advising people update wherever possible.

Additionally, the Lab is updating the Windows installer so that it will verify whether the latest service packs for Windows XP versions have been installed, otherwise viewer installation on XP will be blocked until such time as the relevant service packs have been installed.

The aim here is to help people enjoy improved stability in the SL experience. However, once these new requirements come into effect, they may cause some upset among those affected. How widespread this is liable to be is debatable; the Firestorm team have been running a similar process since the release of Firestorm 4.6.1, and they are reporting minimal complaints from among users.

Group Chat Optimisation

Following the last group chat tests on Aditi, Simon Linden reports that the optimised code has been deployed to a production server this week, but it is liable to be a little while longer before it is deployed to all of the chat servers, and time will be taken to see how it behaves with traffic on the server on which it is currently running.

Group Bans

Baker Linden is back on code merges for the viewer. It is thought he may have hit one or two problems, so assuming he makes the Server Beta meeting on Thursday May 1st, we may find out if this is the case and what is happening.

Experience Keys (/Permissions)

Expectations on what these are liable to be, how they will work, what limitations may be placed on them, and so on, is still running high, particularly after Danger Linden (Don Laabs, Linden Lab’s Senior Director of Product) mentioned them recently. As I’ve covered previously, the experience keys are essentially the culmination of a project which was initially prototyped with the Linden Realms game, and which have since been going through continued development, re-definition and enhancement and which should – hopefully – be appearing in the near future.

Other Items

URL Errors When Connecting to a Region

There is a known issue which can cause issues when the viewer is trying to connect to a region. In these circumstances, the viewer receives URLs containing the host name, but not the required “.agni.lindenlab.com” (e.g. to something like “https://sim10586:12043/cap/…” is received, rather than “https://sim10586.agni.lindenlab.com:12043/cap/…”).

This causes the viewer to fail to connect to the various capabilities using the URL calls, which in turn results in things like mesh load failures, inventory load failures,  L$ failures, and so on, as well as having bad URLs shown for http-in. See also BUG-4704.

Should this happen, the advice to region owners is to contact support, indicating the region where the issues are occurring. The problem can be identified in the viewer log file, which will contain entries similar to the following:

2014-04-29T16:44:05Z INFO: LLCurl::completedRaw: Failed to deserialize LLSD. https://sim10586:12043/cap/01a64236-9a2b-4008-bef0-46a0f7afecae [499]: STATUS_ERROR
2014-04-29T16:44:05Z WARNING: BaseCapabilitiesComplete::errorWithContent: [status:499:] {'reason':'STATUS_ERROR'}
2014-04-29T16:44:05Z INFO: failedSeedCapability: posting to seed https://sim10586:12043/cap/01a64236-9a2b-4008-bef0-46a0f7afecae (retry 23)
2014-04-29T16:44:05Z WARNING: LLURLRequest::smileytongue:rocess_impl: URLRequest Error: 6, Couldn't resolve host name, https://sim10586:12043/cap/01a64236-9a2b-4008-bef0-46a0f7afecae