SL project news week 43/3: viewer status updates

SL Beta Viewer

A new Beta version of the beta viewer was release on the 24/25th, (3.4.1.266251), which has more-or-less the same visible changes as the previous release, except that tcmalloc has been re-enabled as LL seek to determine whether stability issues have been improved or not. However, given the high crash rates experienced with the previous release, Oz Linden is requesting people take time out to give the beta a run. Speaking at the OpenDev meeting on Thursday 25th October, he said, “Run the beta viewer a lot. We think this one will be good, but it takes a lot of usage to find that out and things won’t get unstuck until we do.”

Tcmalloc has been re-enabled with the latest beta release, and this probably means that those using cloud storage such as Microsoft Skydrive and others (such as Cloud Drive – see FIRE-7520 – are liable to encounter issues using it.

The intent at LL still appears to be the complete disabling of Tcmalloc from the viewer code, however, the attempts to do so to date have only met with partial success – the “fix” for MS Skydrive, for example, apparently only worked for some instances, not all.

Commenting on the situation at the OpenDev meeting, Oz said that efforts to completely disable tcmalloc are “taking too long”. As a result, and to prevent work with tcmalloc from adversely affecting attempts to stabilise the viewer, the decision has been taken to move the work on tcmalloc to a separate branch of the viewer code for the time being, with efforts on the beta branch focusing on stability.

In the meantime, code releases remain blocked, pending confirmation the beta release has improved on previous crash rates (which were around 14%).

Mesh Deformer Viewer

As reported in part 2 of this week’s project news, a new version of the Mesh Deformer project viewer appeared over the weekend – 3.4.1.266081, dated October 20th. In addition, Darien Caldwell has made updates to the SL wiki page on the avatar shape XML format which are related to the project.

The updated shape selection options for mesh clothing and human shapes in the mesh upload floater, as seen in the latest version of the Mesh Deformer project viewer

The current thinking is at present that the latest version of the viewer requires wider testing, so if you are involved in mesh clothing / shape creation (humanoid shapes, remember), then you might want to download the viewer and try it yourself. Commenting on the JIRA (STORM-1716), Darien notes:

If people could provide feedback on the use of the custom base shape part specifically, that would be great.

Keep in mind the potential bone length problems outlined above: do these affect your creations negatively when used with custom base shapes?

And is the potential workaround of setting bone lengths to the defaults a problem for your workflow?

Those and any other issues, please post. Thanks 

Initial feedback from those who have tried the viewer have been largely positive, although the issue with custom shapes and certain sliders, as previously reported in these pages (and on the JIRA) remain. Darien has been looking a little deeper into the latter issue, which affects a total of eleven shape sliders, and seems to have found the potential cause:

“The bone length deformation seems to be performed in the function LLRiggedVolume::update(). The deformer is doing its processing before this is reached, and when creating the custom base shape, it’s never run on that custom base shape. So the bone length deformations aren’t included.”

Instead, the current  deformer code assumes that all joints in a shape have the same scale, and that the scale is uniform in all axes. This is fine for a standard shape, but doesn’t work so well with custom shape forms, and wouldn’t work were scaled bones within avatar shapes ever to come about. The solution for this issue is unclear; the current plan is for Darien to write-up her findings for LL, so that someone there can look into the code as well.

The problem: bone length deformation is performed outside of the deformer code, which in turn appears to assume that all shapes use the same, uniform scale for joints. When working with a default shape, neither of these issues present themselves in obvious ways. When working with custom shapes, however, the problems become very evident. In this image a tall, then shape is use to generate an upper body mesh (flesh toned). When uploaded and applied to the shape itself (shown in black), the discrepancies become clear. Had the mesh been correctly deformed according to the shape data, it should more closely match the original shape

Group Services Project Viewer

Baker Linden’s server-side Group Services code for handling very large in-world groups was rolled out to four regions on the main grid on the 25th October, as reported here. However, due to the issues with the beta viewer code as reported above, testing of the new code requires the Group Service project viewer. If you do manage any group(s) with more than 10K members, you may wish to download the project viewer and give things a test on the nominated regions (see link above).

The viewer is available in Windows, Linux and Mac flavours.

Related Links

New Group Services code now live on Agni

Large group loading: a familiar problem

The Group Services project is an attempt to improve the management and editing of large SL groups by replacing the current UDP-based service (which has capacity issues with the size of group lists it can comfortably handle) with a new HTTP-based service.

Oskar Linden has announced that the new code for handling large groups is now live on four regions on the main grid. His announcement reads in part:

These fixes were released to Magnum this morning, but then we found issues in llSensor() that required us to remove the code from a channel as large as Magnum. We have a very small qa channel called the Snack RC Channel that we put the changes on.

 – https://wiki.secondlife.com/wiki/Release_Notes/Second_Life_RC_Snack/12

You will need to be  a member of the group Second Life Beta to access them. You can join that group here: http://world.secondlife.com/group/19657888-576f-83e9-2580-7c3da7c0e4ca

The four Snack test regions are:

Currently, the viewer-side code required to use the new service is only available in a project viewer, which is available as follows:

The viewer side code, as I’ve been reporting in these page, is currently being held-up from more widespread use by crash issues impacting the SL beta viewer code. As such, TPV developers are being encouraged not to merge the code for the above project viewers with their own development viewer code for the time being.

In the meantime, if you manage a large in-world group with more than 10K members, you may wish to download your relevant flavour of the project viewer and try-out the new code on the nominated regions.

Related Links

SL project news week 43/2: SL viewer, mesh, and materials processing

SL Viewer News

Things are not looking good on the official viewer front. As previously reported, the crash rate for the 3.4.1.265898 beta release was unacceptably high when compared with the current release version.

A new beta candidate  – 3.4.1.266073 – was released late on Monday 20th October. As per usual, it will remain on the beta channel for a couple of days once released in order for LL to get a read on performance and crash rates. Some regression testing has started, with the beta code merged back to the viewer-development pipe, but work is really pending on confirming the beta code is relatively stable.

In the meantime, feature changes to the official viewer remain frozen, and even should the new beta release prove stable enough, it is going to take some time for LL to prioritize all of the pending updates (both their own project work such as the Steam updates, HTTP project, Group Services, etc., and contributed updates), start merging them into the code, testing them, and then releasing updates once more.

With regards to overall viewer stability, A series of crash fixes from Firestorm (officially viewed by Linden Lab as the most stable viewer connecting to SL) have been contributed to LL and are awaiting regression testing in order to confirm their effectiveness with the LL code.

Mesh Deformer

A new version of the Mesh Deformer project viewer appeared over the weekend – 3.4.1.266081, dated October 20th. This has the revisions Darien Caldwell has made to the uploader and custom shapes, although she is still waiting on feedback from Qarl on the matter of the sliders / bone armatures which are deformed by both the viewer and the deformer, as mentioned in part 2 of my week 42 update.

The new version should hopefully include a small bug fix when selecting the Default Male shape (which would act as if the Default female shape were selected).

Materials Processing

Materials processing: using a normal and diffuse (texture) map to generate a 3D effect on an in-world wall – coming to SL in the near future

Speaking at the OpenDev meeting on Monday 22nd October, Oz Linden indicated that the specification for the revised build floater in the viewer – needed to handle picking, rotating and offsetting normal and specular maps in the same way as can be done with textures (diffuse maps) at present.

The specification for the floater has been written in-house at LL, but the work is being undertaken for LL by a third-party viewer developer who volunteered to carry out the required work on LL’s behalf. This work will initially focus on getting a shell for the floater built, as there are a number of things that are being worked on within the viewer with regards to making parameters actually visible on surfaces.

Discussing progress at the Content Creation Improvement Informal User Group on Tuesday 23rd October, Geenz Spad outlined some more of the upcoming capabilities, as defined by the initial release specification document.

“We have normal maps with specular exponent stored in their alpha channel, specular maps will have environment masks stored in their alpha channel [so, for example, mesh won’t require seperate faces in order to have different levels of shininess on different parts of it], and for diffuse maps you’ll be able to select to some limited degree what its alpha represents (currently, none, alpha blending, alpha masking, and emissive mask).” He went on, “There’s a few additional knobs people will be able to mess with as well, such as specular exponent (combines with the normal map’s exponent map in its alpha), specular color (combines with the specular map’s color), and environment intensity (combines with the specular map’s alpha).”

However, custom environment maps – such as used with custom reflections – won’t initially be supported, as has been previously indicated. With the initial release specification now locked – but not currently open to public reading, that will occur nearer the time LL are ready to make a further announcement on materials – any additional functionality is being looked upon as being either for a further release or requiring a programmable system to implement.

Whether there will be further updates to the system remains to be seen, but it appears those involved in the project, both inside and outside of LL are reasonably confident there will be. “Materials is something that you can likely expect to receive several updates over time,” Geenz commented. “It’s something that’ll evolve from a fixed system into something a fair bit more sophisticated eventually.”

Related Links

SL project news week 43/1: Server updates and llHTTPRequest

SL Server Updates

A brief start-of week update with an important item on the LSL llHTTPRequest function.

Tuesday October 23rd saw an update to the main channel which should have minimal impact on things, “It’s a change that should make simulators run better on our new hardware,” Simon Linden explained at the Simulator User Group meeting on the 23rd.

Wednesday 24th, as previously indicated, should see the RC channels updated as follows:

  • Magnum should receive bug fixes together with Baker Linden’s Group Services project code (the viewer side of which is still blocked)
  • LeTigre should receive further updates for the new Havok code (which presumably include fixes for the crash loop situation Maestro reported in the Server Beta UG meeting (see above)
  • BlueSteel should receive, “Some more invisible changes that should help us deal with some problems like full disks that make servers very unhappy.”

Details on the deployments are, as usual, posted in the Second Life Server section of the Technology forum.

Third-party Web Caching and llHTTPREQUEST

Kelly Linden indicated that the updates to LeTigre in week 42 had some library updates. One of these updates was to the cURL library which changed its behavior specifically around caching.

Until now, outgoing requests on have a Pragma: no-cache header in them, because cURL added this to all requests, and thus ensured fresh data was returned. The change made to the cURL library on LeTigre means that this is no longer the case, so if the third-party web server has caching enabled, any outgoing llHTTPRequest might return previously cached results from the server, rather than fresh data.

Kelly noted that, “Systems that are most likely to be affected are those that frequently hit the exact same URL and expect the data to change. Maybe they are getting a counter or checking on something’s status, leading to problems with the likes of breedables dying, and so on.”

A workaround for this has been implemented for the llHTTPRequest in the form of a HTTP_CUSTOM_HEADER flag, which enables Pragma: no-cache to be specified manually.

While there have currently been no reports of this or similar happening so far, LL are continuing to discuss the potential impact. Further, it is expected that the , Kelly continued, “If anyone uses or develops systems they think *might* be affected please give them a try on LeTigre this week and let me know. Or if you know others that might, encourage them to test on LeTigre this week. Thanks.”

So, if you have created a product which uses an external web server for updates in the manner described above, etc. (or know anyone that does), you may want to test behaviours on a LeTigre region to see if your product is impacted prior to this change rolling out further across the grid.

SL Issues

Network Traffic and Sim Lag / Crash

This issue has been going on since the start of the month, and appears to affect regions with large numbers of people.

A bug report was raised on the issue (BUG-355), and has been imported by LL as a MAINT issue (MAINT-1682). However, there has been no feedback from LL as to the underlying cause, although investigations are continuing.

CHUI: no, not a Wookie, a viewer from LL – with feedback requested

Linden Lab have launched, somewhat unexpectedly, a new project viewer, called CHUI. While sounding like a character from Star Wars (CHU-EE, geddit? *Ahem*. Sorry), it stands for Communications Hub User Interface. The blog post states:

With so many ways for users to communicate with one another in Second Life, there are quite a few communications tools in the Viewer. To make it easier to find, learn and use these tools, today we released a project Viewer that introduces CHUI (Communications Hub User Interface). In addition to bringing most of these communications tools “under one roof,” CHUI also introduces some new and improved features.

Among the listed features are:

  • The Conversation Log: providing you have enabled the option to save chat and IM logs to your computer, this allows you to open the entire history of a conversation with another user held in the past 30 days directly in your viewer, or review off-line IMs received from both friends and non-friends
  • Expanded conference calls: with CHUI it is possible to add people to a conference call after it’s started, or add someone to an existing one-on-one IM session
  • The ability to easily move your voice connection between open conversations including nearby chat, private IM, conference chat or group chat. Click the “add voice” button on any conversation to move your voice connection to that conversation. Click the “hang up” button and your voice connection is returned to nearby chat.
  • The ability to access chat preferences in a single click from the Conversation window
  • Change the volume of a single person’s voice by simply clicking on that person’s speaking icon in the Conversations window
  • A multi-line chat entry box which expands as you type.
CHUI Conversation Log

These features primarily found in two floaters: Conversations Log and a revised Conversations floater. The Conversations Log window lists all recent and past conversations, allowing them to be to scrolled through and opened for reading. As I’ve only jut started using the project viewer, I’ve actually not investigated this in-depth. Clicking on a listed conversation will open in the Conversations floater, and the Conversation Log contains two buttons for sorting the listed conversations (by name, by date, etc.), and a gear cog button for access various options – start an IM, enter a voice call, view profile, etc., for a selected conversation in the list.

For those who use TPVs with tabbed IM capabilities, the revised Conversations floater will look remarkably familiar,  bringing as it does local chat and all IM conversations into a single floater panel. Any conversations in the Conversation Loge will also open here as well.

The Conversations floater

The panel includes a number of buttons. These again allow conversation to be sorted, closed individually, etc., and also include a number of additional options:

Add someone else to an existing IM conversation, and establish a conference call. This will open the Choose Resident Floater, allowing you to pick a friend, someone nearby or search for someone.

Start a Voice conversation with a person or hang-up from a Voice conversation (the icon will change on the button, depending on the status of the call)

Open the Choose Resident floater to select someone with whom to start an IM or Voice conversation.

Break-out any conversation into its own floater.

The Conversations floater can also be compacted down into one of three sizes, using the left / right double chevron arrows. These help reduce the amount of space the floater takes up on your screen when not actively in use. It can be expanded either using these buttons or using the right-pointing arrows next to the names in the conversations list.

The three compact views of the Conversations floater

Overall, this is a significant attempt to centralise in-world communications, and there are some nice features here, particularly in the extended Voice options.

For Linden Lab, this is very much experimental, as noted in the blog post itself, and they are asking for people’s feedback on the features:

We’ve been testing CHUI inside Linden Lab for some time, but any major redesign requires a lot of people using it to make it as smooth and useful as it can be. This is where you come in.

Please think about these questions as you use the CHUI project viewer:

  • Are the new features useful?
  • Do the functions you commonly use seem more streamlined, or do they require more clicks than before?
  • Are all of the functions, both old and new, easy to find?

We’ll ask you to complete a survey in approximately one week to gather your thoughts on these questions.

There is a publicly accessible JIRA (https://jira.secondlife.com/browse/chuibug) available for the viewer, and if you do try it out and find a bug, LL request you report it there.

Also tucked away in the blog post is news that blocked users and objects can now be viewed from within the People floater, rather than via a separate menu option, and can also be unblocked from here via the right-click context menu.

Related Links

SL project news: week 42 / 3 – server news

There is a lot going-on server-wise at the moment, so best to break it down by heading.

Server Rebalancing

The Lab is currently engaged in a rebalancing exercise in an attempt to put neighbouring regions on the same server and generally do a logical organizing of the grid to help improve various aspects of performance. Speaking at the Server Beta User Group on Thursday 18th October, Oskar Linden explained that there has been a lot of moving around (in terms of regions) within and between the Lab’s co-location facilities, and so the re-balancing is warranted and needed.

This work takes time – a rebalancing operation earlier this year took around 6 weeks to complete. It requires that regions are organized into groups and then generally moved twice: the first time to a temporary sever, the second to the target machine, each move requiring a reboot, so people may notice additional and unexpected restarts to regions they are in as the work progresses. Two moves are required because the server topology is so tight, it often isn’t possible to move regions directly from one server to a target server, so an intermediary is required.

While this work will take time to complete, the result should be improvements in stability and performance with the likes of teleports, etc, and even improved region crossings.

More on the Server Deployments for Week 42

  • Main channel: Oskar provided some more information on the main channel update of Tuesday 16th October, saying, “The main channel got a tweak this week, but it was a really small change, and no sim code got changed. We recognised that we had some inefficient SQL queries where large groups were concerned, so we optimised them, and the effect was quite noticeable. The databases are more responsive [and] this helps at all levels.” He went on to clarify that these changes were not Baker Linden’s Group Services code changes, after some in the meeting appeared to think this might have been the case
  • BlueSteel received the updates which were tested in the network pile-on test in week 42. At the time, I commented that teleporting seemed a lot faster, but that might have been a placebo effect of being on Aditi. It was. Commenting on the test, Oskar said, “There were no simulator changes in that test code. We were just testing backend tweaks.”
  • Magnum received no update per se, as previously reported, but was merged with trunk and then redeployed
  • LeTigre received the biggest update, which included new LSL functions ad updates, and most importantly of all, a new version of Havok (see below). Of the LSL functions, Maestro had a warning about the new OBJECT_PATHFINDING_TYPE parameter in the pathfinding command llGetObjectDetails, “We misspelled a constant, OPT_UNKNOWN, so we plan to fix that.” The fix will probably be next week.

Havok

As mentioned above, Havok on LeTigre was updated to version 2012.1. The update enables Havok’s built-in terrain optimisation and should lead to improved performance as a result of the physics shape of the terrain being simplified. Prior to the deployment, there were concerns that it would lead to issues with mesh vehicles trying to cross between regions running different version fo Havok, as has previously been the case.

As reported in part one of this update, these concerns led to Andrew Linden contacting the deployment team in LL to check whether it would be possible to ensure none of the Blake Sea regions remained on LeTigre while two versions of Havok running on the grid to help alleviate at least some of the pain people would feel when using mesh vehicles there. This apparently happened, whether it was before or after the deployment is unclear, as some people did report issues following the roll-out. There was also a little confusion as to what had been swapped where.

At the Server Beta meeting, Oskar gave the impression that all Blake Sea regions were on LeTigre. However, at the Simulator User Group meeting on Friday 19th October, Andrew Linden indicated that records showed none of the Blake Sea regions are running on LeTigre, although they are spread across the other channels. Given that there were (according to Andrew) around six Blake Sea regions running on LeTigre to start with, it would appear to make sense that they have been rotated off to another channel, rather than attempting to rotate all of Blake Sea on to LeTigre.

Please use the page numbers below left to continue reading this article