Tag: Group Services

SL projects update week 40 / 2

Posted on by Inara Pey

Server Deploys

As many are aware, there was a major error in this week’s LeTigre Release Channel deploy. Apparently, the root cause of the problem lay in the server-side prim account code, which Simon Linden describes as having “blown up” on the LeTigre RC channel. This resulted in a large number of items (including partial builds) being returned to people’s inventories as a result of regions being seen as “full”. The problem required a two-stage recovery:

  • LeTigre regions were rolled back to a state prior to the faulty deployment, and were then updated with the BlueSteel code also deployed on Wednesday October 3rd. This helped to determine the extent of the damage (a total of some 1200 regions)
  • The regions damaged by the land impact miscalculation were then restore to a state prior to the roll-out of the original faulty LeTigre code. These had to be restored manually, which took a considerable time

There is further post-mortem work going to to try and discover why this error did not reveal itself when the code deployed to LeTigre was being tested on Aditi, and whether there is anything specific to the regions impacted by the error which may have triggered it. Thought is now also being given to managing large scale region restorations, despite this being the first time there has been such a massive issue of this kind occurring across the grid.

Current RC plans for next week call for the same maintenance release to be made to all three RC channels, which Simon Linden describes as, “Mostly internal changes but [which] does include a minor update for the physics engine library … It’s almost all updating libraries … we’ve been using a fairly old set of compilers and such to make some of the development builds of the servers, and this brings us to more recent code.” Further details on the deploy should be available next week in the Second Life Server section of the Technology forum.

SL Viewer

As indicated in part one of this report earlier this week, problems have continued with the Beta viewer code and high crash rates. Work has been ongoing to try and locate the probable cause(s), some of which included the temporary return of tcmalloc. While not actually a cause of the crash issues, having tcmalloc disabled was affecting efforts to reproduce the problems. a beta release was made on the 3rd/4th October (3.4.1.265434), which is proving to be a lot more stable than previous versions, and which happens to have tcmalloc enabled.

The current plan is for a further beta release to be made, most likely on Monday 8th October, which should see tcmalloc turned off once more (if not removed). Should this also prove to be stable, the fixes it contains will be merged back into the development viewer code, and this will clear the way for clearing the backlog of code merges for both the beta and development viewers. It may also see a further 3.4.1 release version of the viewer being made.

Among the projects awaiting merging into the development and beta viewer code are:

  • The Steam support changes, which have been available within a development viewer stream, and which are described as “mostly cosmetic”. There is apparently a version of the viewer on Steam, but it is not available for general viewing / download, and is presumably there for testing purposes
  • Monty Linden’s HTTP library (texture fetch) code
  • Baker Linden’s Group Services project code
  • Apple OSX 10.8 Mountain Lion support work, including gatekeeper compatibility
  • Bug fixes and further regionalisation work.

Previous plans for these releases called for them to be made under the 3.4.2 code base. While this wasn’t discussed at the TPV/Dev meeting, one assumes this is still the case. However, speaking at the TPV Dev meeting on Friday October 5th, Oz Linden indicated that the order, etc., in which waiting merges will be cleared hasn’t been fully defined, and will be the subject of internal conversations next week at the Lab.

Avatar Baking Project

Bake fail: a familiar problem for many

There is still no major news on this project, although work is continuing both on the viewer and on the server code.

The plan remains to provide TPV developers with access to the viewer code at least 8 weeks ahead of any initial deployment of the server-side code to an Agni release channel. This is to allow TPVs time to merge the code into their viewers and participate in ongoing testing of the new service.There is a possibility that that viewer code will be available sufficiently well ahead of things in order for TPVs to be able to use it alongside the testing on Aditi (beta grid), depending on the status of the beta grid tests and how development of the viewer code progresses.

Please use the page numbers below left to continue reading this report

Pages: 1 2

SL projects update: week 40 / 1

Posted on by Inara Pey

Update 3rd October: The RC roll-outs were somewhat different than indicated by Simon. Magnum apparently received the back-end configuration work for new and future hardware, and BlueSteel received the maintenance update. The Group Services code & maintenance release was targeted at LeTigre, however a showstopper issue means that these regions are now in a state of flux. A comment from Oskar Linden reads: “LeTigre regions exhibited issues that necessitated a rollback. The regions were rolled forward and are on the same code that is on BlueSteel. Affected regions are getting a simstate rollback” . 

Server Deploys

There was no main channel roll-out this week, as expected, following the cancellation of last week’s RC channel deploys.

Wednesday 3rd October will see the three RC channel deploys originally planned for last week:

  • Back-end configuration work designed to help SL run better on new and future hardware – this should be deployed to BlueSteel
  • A maintenance release, which includes has Baker Linden’s Group Services project – this will be deployed to Magnum
  • The third is described by Simon Linden as being, “Very similar to what we have today, with a fix for some future compatibility coming down the pipe. It’s nothing really exciting, but required so things won’t break.” – this should be deployed to LeTigre.

The order of the releases is not clear at the time of writing, and is based on Simon Linden’s comments at the Simulator User Group.Confirmation / updates to the plan should be made available via the Server Deployment thread in the forums.

SL Viewer Releases

Things remain slow due to on-going crash / possible memory leak issues, as reported in my last mini-update.

Group Services Project

The Group Services project is an attempt to improve the management and editing of large SL groups by replacing the current UDP-based service (which has capacity issues with the size of group lists it can comfortably handle) with a new HTTP-based service. The project viewer for this is already available (for Windows, Linux and OSX.). It had been hoped that the viewer code might reach a 3.4.2 beta release during this week, however due to continuing issues with the current beta viewer code, this now seems unlikely.

The initial server code for this will be deployed to the Magnum RC channel, as mentioned above, on Wednesday 3rd October.

Materials Processing

The final feature set for the first release of materials processing is now more-or-less complete, and it should hopefully be announced nearer the time for beta testing. A number of regions on Aditi have been prepared for beta testing, and details on these will be released when the initial project viewer is ready for release.

Options to be used by Normal and Specular Maps, which will need to be provisioned in the Build floater

The initial feature set will include the ability to set parameters (rotation, offset, etc.), for normal and specular maps as well as diffuse (texture) maps. Oz originally hinted this was the aim a few weeks ago, although Geenz indicated it was only hoped these capabilities would be included, rather than it being definite at that time. This means that the build floater for the project viewer will be somewhat different from most people who build are familiar with, as it must include a number of additional options (see right). However, what is being considered is not a complete rebuild of the Build floater.

Commenting on this aspect of the work at the Content Creators’ Improvement Informal User Group, Oz Linden said, “We’ve got a strawman design in internal review… will have a version to look at soon, I think.”

The design has had input from a number of builders. Some of these are from within LL, some of them users, and will probably be left unchanged until further experience is gained in its use. Whether this means the viewer remaining unchanged between the beta programme and the release of materials processing on Agni, or whether changes are made between the beta and the release, remains to be seen. As Oz went on to comment, “It’s a very difficult problem, and we tried to meet three sometimes-conflicting goals: do what needs to be done; keep things familiar; make the things you have to change better in the process. I think we did pretty well…”

Obviously, TPVs will have access to the build UI code once it is available in LL’s accessible repositories, and they’ll doubtless look at the code in terms of how best to integrate it into the look and feel of their own viewers.

There are still no firm dates for the project in terms of beta commencement, etc., but Oz reiterated that the project will follow the familiar course, with an Aditi beta, followed by a release to one of the RC channels on Agni, prior to an eventual full roll-out.

Related Links

With thanks to JayR for the simulator UG meeting transcript.

SL projects update, September 26th

Posted on by Inara Pey
Update 28th September: Please also refer to an update post on some of the projects / news given here.

SL Viewer Status Updates

Linden Lab have been working hard on a range of viewer-related issues, notably crash rates and memory leaks, which have slowed the viewer update a release process up over the last few weeks. In terms of memory leaks, tcmalloc has been identified as the culprit, with Linden Lab deciding that dropping it is “probably a good idea”, according to Oz (tcmalloc has previously been implicated in crashes linked to the use of things like Microsoft’s Skydrive). There have also been an issues with LL’s statistics system which have meant that the viewer hasn’t necessarily been accurately tracked in terms of crash rates, etc.

Beta Releases

As it stands, LL hope to have the blocks on the various code merges removed during this week, which should see a rapid series of beta releases coming down the pipe. This work commenced with an initial 3.4.1 beta release (3.4.1.265134) emerging on September 24th. It will be followed by around three or so additional and rapid 3.4.1 build iterations aimed at confirming the viewer’s stability and at replacing various fixes which had previously been removed from the viewer code while trying to identify the causes for the viewer crashing / suffering memory leaks. It is expected that each of these iterations will be on the beta release build channel for a couple of days, prior to being replaced. Following these there will be a series of project updates, the first of which gatekeeper compliance project, which is also targeted for a 3.4.1 release build.

Project-related Releases

Once the stability of the beta viewer has been confirmed, it is anticipated that project-related code will be merged into the viewer, most likely starting with 3.4.2 builds. Among the releases planned for 3.4.2 is Monty Linden’s HTTP Library Services and Baker Linden’s Group Services code. These are currently targeted to reach the beta build channel in week 40 (week commencing Monday October 5th).

These releases will at some point include the Steam updates currently in a Development branch as well, which might in turn mean that Second Life could be ready to appear on Steam in the very near future, once these updates have reached a release version of the viewer.

Account creation prompt: heading for the beta viewer

Group Services Project

The Group Services project is an attempt to improve the management and editing of large SL groups by replacing the current UDP-based service (which has capacity issues with the size of group lists it can comfortably handle) with a new HTTP-based service. The project viewer for this is already available (for Windows, Linux and OSX.), however, as mentioned above, the current plan is to get this into the 3.4.2 build stream alongside the HTTP textures project, possibly in week 40.

Originally, the server code for this project was due to have been rolled to the RC channels during week 38, (week commencing September 17th), but the channel deploys were postponed after QA issues were found. As a consequence, the roll-out was due to take place on Wednesday 26th September, but has again been postponed.

There has been some confusion as to the aim of this project, with some people believing it is focused on fixing group chat issues such as  lag and chat failing to start. This is not the case at all; as stated above, the project is aimed at improving the management and editing of large groups (10K+) through the use of a new HTTP service.

HTTP Library Services

As indicated above, the first phase of this work, covering a new texture fetch service, should be appearing in a 3.4.2 beta release of the viewer in the near future.

HTTP Libraries project viewer: improved texture loading and rezzing

Please use the page numbers below left to continue reading this article

Pages: 1 2

Upcoming SL projects update

Posted on by Inara Pey

There is a lot going on in terms of projects and development work in Second Life. The following is a further update on a number of key projects I’ve been following and reporting upon through the pages of this blog.

Group Services Project

The Group Services project is an attempt to improve the management and editing of large SL groups by replacing the current UDP-based service (which has capacity issues with the size of group lists it can comfortably handle) with a new HTTP-based service. The project viewer for this is already available (for Windows, Linux and OSX.), as I reported last week.

This week sees the server-side code rolled-out to all three RC channels (Magnum, LeTigre and BlueSteel), allowing the project viewer to be tested in handling very large groups (significantly larger than are available on Aditi). Note that those running viewers without the new code on any RC region will be unaffected, as they will continue to access the current UDP service.

There are still no timescales as to how long the testing of the service will last (“It’ll take as long as it takes,” Baker commented recently), or when the viewer code will progress beyond the project viewer. However, a number of things should be noted in reference to the eventual roll-out:

  • The viewer code is not being made back-compatible with V1 code by the Lab. Therefore, TPV developers using the V1 code base will have to backport the code themselves in order to use the new service
  • The initial HTTP service roll-out does not include any data compression. This means there will still be some delay in downloading member lists for very large groups with tens of thousands of members
  • Once the new service is rolled-out, which service is used is entirely transparent to the user. If a viewer with the new code is running on a region which has the server-side HTTP service, it will connect to that service. If it is on a region using the older UDP service, it will connect to that service instead
  • Once the HTTP service is fully deployed, viewers which do not implement the viewer-side code will still be able to access groups with member lists up to 10K in size via the UDP service until such time as it is switched off (which will not occur for some time after the HTTP service has been rolled-out). However, attempts to access groups with lists larger than 10K will fail.

Interest Lists and Object Caching

It’s been a while since I’ve reported on Interest Lists Object Caching, which forms a part of the Shining Project.

To recap: when you enter a region at the moment, your viewer receives a huge amount of information on what requires updating, much of it relating to things you can’t even see from your position in the region. This information is received in no particular order, with the familiar result that things appear to rez in your view in a totally random order. Not only that, but the chances are that if you’ve previously visited the region, much of the information being sent to your viewer is already locally cached – but is being ignored. The focus of this project is to both optimise the data being sent to the viewer and the information already cached on the viewer with the aim of significantly improving object rezzing times in terms of speed and order (so objects closer to you rez before those further away, for examples).

Object caching and interest list changes: easing the pain of random rezzing

Andrew Linden had hoped the project would be going to QA this week ready for roll-out to one or more RC channels in the near future, but some last-minute problems popped-up and have delayed things until he can get them sorted out. In the meantime, the code has been deployed to a number of regions on Aditi, and Andrew plans to, “Try to throw a pile of test avatars at it to stress it out. Later this week.” No viewer-side changes are associated with this work.

Materials Processing

Work continues on the project to bring materials processing to Second Life. Last week, it appeared as if the new materials – normal and specular maps – would have their own rotation and positioning options independent of any texture (diffuse) map. This week, it appears that this is the hoped for situation, but the matter is still open to question – which goes to show how fluid the project is.

The new capabilities require changes to the rendering pipeline, and details have been released on some of what this entails.

In order to work, normal and specular maps require what is referred to as per-pixel lighting (as noted in the original blog post on the subject). As such, there has been a debate on whether it would be better to develop a per-pixel lighting framework within the viewer, or work to make the current deferred rendering system more accessible to per-pixel lighting capabilities. As the latter approach will allow getting materials processing working within SL sooner than would otherwise be the case, it has been decided to go that route. Thus work is focused on making the current lighting system more configurable and able to better handle a broad range of material types (metallic, matte, plastic, etc.), together with adding support for both per-object and per material shading differences.

However, a dedicated per-pixel lighting framework does offer advantages of its own, and as such is being considered as a possible future extension to the project, which may be implemented at some point down the road. One such advantage is that could potentially be run in a non-deferred mode, which might lightening the load on older graphics systems.

Please use the page numbers below to continue reading

Pages: 1 2

Recent SL viewer activities

Posted on by Inara Pey

It’s been a while since I’ve reviewed any of the official SL viewers from LL, so here’s a quick round-up of recent releases.

New Log-in / Account Creation Prompt

The new account creation prompt, displayed if the viewer does not locate any user settings files on a computer, and which first appeared in the 3.4.1.263582 release (August 16th), now looks to be the default option for all development / project viewers. It is part of both the most recent Mesh Deformer project viewer (3.4.1.264215, August 31st), and the new HTTP Group Services project viewer (3.4.1.264495, September 7th). However, it has yet to filter through to either the Beta or release versions of the Viewer.

Account creation prompt: now standard on all development / project releases of the SL viewer released since August 16th (click to enlarge)

Mesh Deformer Project

August 31st saw a new release of the Mesh Deformer (3.4.1.264215), which includes a revised mesh uploader floater with deformation options for the male and female shape.

New deformation options

According to Nalates Urriah, the new options invalidate all test items so far provided for the project, and new samples are now required, although no comments to this effect appear to have been made on the JIRA or elsewhere, so they may have been confined to a user group meeting. Details on how to provide test items can be found in Oz’s forum post on the matter. The JIRA (STORM-1716) for this project is still open for viewing and comment.

Group Services Project Viewer

As noted this week, there is now a Group Services (group management) project viewer available for testing the new HTTP group management service. The server-side of this project has yet to be rolled-out to Aditi, so it cannot be tested as yet. However, Baker Linden, who is developing the service, is apparently updating the JIRA, SVC-4968 (which is still publicly viewable) with the project status, and has indicated he’ll post when the server-side elements are available for testing.

The viewer is available in Windows, Linux and OSX flavours.

HTTP Libraries Viewer

The HTTP Libraries project viewer (3.3.3.262585) appeared on July 27th. This project, which Monty Linden is driving, is currently aimed at improving texture downloading and rezzing as a part of the Shining project.

HTTP Libraries project viewer: improved texture loading and rezzing

Texture loading / rezzing would appear to be significantly faster on this viewer compared with other offerings, although there also appear to be what might be placebo effects associated with it. Some people have reported that floaters, etc., seem to load more slowly, and some have reported various performance improvements outside of the HTTP library changes.

Beta Viewer and Release Viewers

The Beta viewer (3.4.0.264445 at the time of writing) continues to be focused on pathfinding, with fixes and updates going into it on a weekly basis – which is why the pathfinding tools have yet to release a release version of the viewer.  The removal of JIRA numbers from the release notes now means that tracking issues previously being watched is that much harder (even if the JIRA themselves are no longer accessible, having the JIRA numbers still visible facilities easier identification of issues being specifically tracked).

Similarly, the release viewer (3.3.4.264214) appears to be focused on bug fixes and general improvements, with the release notes currently benefiting from the retention of JIRA numbers, making scanning for specific fixes easier.

Performance

I carried out basic performance tests on the viewers listed above using Dimrill Dale as my sample sim, during a period when there were the same number of avatars in the region (5, including myself). Tests were carried out in the same location on the region, looking in the same direction and with the same viewer settings (e.g. Graphics on high, Draw Distance set to 260m, using default time-of-day, with deferred disabled / with deferred enabled and lighting set to Sun/Moon+projectors, etc.). While all such tests are rough-and-ready, these did tend to show that all of the viewers offer the same performance on my default PC (see the sidebar panel on the right of this blog’s home page for system details). My results were:

  • Non-deferred: 18-20fps
  • Deferred with Sun/Moon+ projectors: 8-10fps

Similar figures were also obtained using the current Firestorm and Exodus viewers, although with deferred enabled and Sun/Moon+projectors active, Firestorm was slightly down at an average of 17-18fps, the other viewers being closer to an average of 19-20fps.

Group management project viewer released

Posted on by Inara Pey

As I recently noted, Baker Linden has been working on the large group management / editing issues, developing a new HTTP-based service to replace the current UDP service which has significant issues handling groups with more than 10-11K members. At the TPV/Developer meeting on the 24th August, he indicated that a project viewer would be available in the near future.

On Friday September 7th, he updated the JIRA on the issue with notes that the project viewer is now available for Windows, Linux and OSX.

In commenting on the JIRA (SVC-4968), Baker notes that the server-side code for the new service has yet to be deployed to Aditi, where initial testing will take place, and adds that he’ll be providing an update on the status of the server code once the situation has been clarified. He goes on to add:

There may be some issues during testing. When getting the member list of a large group, other info (group title, group info, etc.) may not properly load. This is an issue with the speed of Aditi’s SQL server and shouldn’t occur once live on Agni. To receive the rest of the data, wait for the member list to appear (this can be upwards of a few minutes), go back to the My Groups panel of the people floater and view the group profile again. The query will be cached this time, and the member list will appear quicker than it did before (depending on your connection speed). The rest of the information should be received this time.

If you find any problems while testing, please send me a message in-world (on Agni).

Large group loading: part of the group management problem

As noted in my previous report, in the first implementation, the data will be uncompressed. This means there will still be some delays in group loading (Baker previously estimated that a 40K member group is around 5Mb in size and could take up to a few minutes to download, depending on someone’s connection speed). Data compression is being looked at for a future release, although as noted in the comments on my last article, some are wondering why paging group data isn’t being implemented (does the viewer API support it?).

Another point of note is that the new service is not compatible with V1 code, so adoption by V1-based viewers is liable to require some backporting. This is important, as once the new HTTP service is rolled out, the older, more limited UDP service will be capped at groups containing 10,000 members – larger groups will not function.

There is still no definitive time scale for the roll-out of the new service. However, it seems likely that once available on Aditi, the server code will remain there for testing for at least a couple of weeks prior to it being added to a RC channel on the main grid. How long the testing period across both will be is open to question, and a lot will depend on feedback as to how well the new service performs.