Category: 2012 Project Updates

SL projects update week 41 / 2

Posted on by Inara Pey

This item is a follow-on from part one, published earlier this week.

More Server News

At the Thursday Beta Grid User Group meeting (Thursday October 11th), and prior to the network optimisation tests, Oskar gave further news in the  serve deploys for week 41. On Tuesday 9th October, the main channel received code previously on BlueSteel, which in keeping with Simon Linden’s comments at the Monday Sim / Server UG meeting, Oskar referred to as being, “A pretty small release, just some server crash mode fixes; stability ++.”

On Wednesday October 10th, BlueSteel and LeTigre received a fix to some database queries that were really slow when accessing really large groups (note these were not Baker Linden’s Group Services code, that is being looked at as a deployment in week 42).

Monday 16th October may see some restarts on the grid in order to shuffle some regions onto new hardware, with the servers having more and faster CPU cores, which will increase the number of simulators running on the new servers, but they’ll be running on faster CPU cores.

Interest Lists and Object Caching

The short-version update for this comes from Andrew Linden, speaking at the Server Group meeting on Friday 12th October, “I thought I’d have something working this week… it isn’t quite working right. You can see it not working on Ahern on Aditi…” (!)

Interest list changes: easing the pain of random rezzing

He went on more seriously to explain that while the new code is working correctly for the most part, and that rezzing orders should be improved / faster, there are some problems with objects which should be in view of an avatar not showing up and a major issue around teleporting into high ground.

When the latter happens, you effectively arrive “underground” (presumably at the default “ground level” for the simulator  – 21 metres in the case of unterraformed land). The simulator then calculates where you should be and moves your avatar appropriately. With the new code, this has the effect of breaking the server’s notion of the camera – where it is and what it can see – which is used to figure out what objects to send to the viewer. This means that the camera itself cannot be moved or updated.

There have been some performance tests on an older version of the code, which have been mixed, as Andrew also explained, “here were two performance tests run on an earlier version. One test (mostly empty region with about 30 avatars running around) showed a slight decrease in performance… about 5% worse. Another test (crowd of avatars NOT looking at a pile of dynamic objects behind them) showed about 40% improvement (less time spent running the interest list). So I went back to the code to try to fix the first test, and I think I’ve got something that will be as good or better all around.”

The code will also see changes as to how the camera behaves and in the resultant level of detail. Andrew is currently working on limiting the distance the camera cam be moved away from the avatar. Note this is not limiting Draw Distance, but limiting the distance the camera can be freely moved independently of the avatar. He’s considering 128 metres to be the likely range. There are two reasons for this.One is to prevent the camera wandering into regions which are more than one neighbouring region away, the other is because as the camera moves laterally, detail levels degrade, because object detail is tied to the avatar’s position (hence why, when you zoom a great distance, buildings and objects may only appear to partially rez, etc.). Under the new system, object detail will be tied to the camera, so that little degradation is experienced. However, in order for this to work, the camera must be kept within a reasonable distance of the avatar; if it is moved too far, the detail will start to degrade once more (presumably because of the volume of data the viewer is trying to handle).

Mesh Deformer

On Thursday 11th October at the Open/Dev meeting, Darien Caldwell outlined her ideas for using base shape info exported from Second Life when uploading rigged meshes.

If this works, it will essentially mean that rather than being restricted to using a default base female or male shape when uploading rigged meshes, creators will be able to download a human shape as an XML file (permissions allowing), and then specify this shape when uploading rigged meshes.  The basic code for handling the upload with specific avatar shape information has already been added to the deformer by Qarl Fizz, so Darien is focusing on the best way to use it, her work going into a fork of the existing Mesh Defromer project viewer.

Avatar shapes can currently be exported from a viewer via DEVELOP -> AVATAR -> APPEARANCE TO XML (again subject to the permissions system). This saves the avatar shape data as an XML file, which contains the settings from the appearance sliders, and which is automatically saved to your computer (generally to  C:\Users[USERNAME]\AppData\Roaming\SecondLife\user_settings for Windows).

To associate an avatar shape .XML file with a mesh, Darien is proposing a further revision to the mesh uploader floater, and has provided an early mock-up as to how it might look.

New option to associate an avatar shape XML file with a mesh on upload (image courtesy of Darien Caldwell)

More work is required the flesh-out this idea, including, as Oz noted at the Open/Dev meeting, making the shape export option more obvious for people to use, which will more than likely see it moved out of the Develop menu, wherein it is currently nested. The .XML file itself is not suitable for use directly in most 3D modelling programmes, so how the exported data might be used with these when creating mesh items remains to be seen. nevertheless, if successful, Darien’s approach may offer a more fine-tuned solution to developing mesh clothing to a range of shapes.

Other items

Viewer and FMOD

The use of FMOD has been the subject of much discussion within the TPV/Dev meetings of late. FMOD is used within the sound system for the Viewer, and until now, Linden Lab has provided a script which pulls library files from an FMOD repository for use in viewer builds. However, following what appears to have been a clean-up of their archives, FMOD have removed the some of the legacy files required for this, as reported in JIRA OPEN-150.

Some viewer developers have already started using FMODex within their builds (e.g Singularity 1.7.0+), which also addresses issues with sound quality as well. Other TPVs are looking at possibly integrating this work into their builds.

It currently appears as though Linden Lab themselves are looking to integrate FMODex, as they see this very much as something which needs to be addressed. Speaking at the TPV/Dev meeting on Friday 5th October, Oz Linden stated: “I got around to forwarding the JIRA on that to our engineering manager for Second Life, and he agreed with me that it is something we should definitely do something about. I’m not sure what the time-table on that will be, it’s going to go into the hopper for the next ‘Things we should do something about, what priority are they compared to all the other things we should do something about’ meeting, which happens weekly.” While openAL has also been suggested as an alternative, it does seem more likely that FMODex will be adopted, something which was hinted at by Oz when talking at the Open/Dev meeting on Thursday 11th October.

Teleport Timeouts

Baker Linden has been looking into the issue of teleport timeouts, and has managed to pin down one cause as a reproducible bug. He’s not sure as to whether it can be fixed, and is currently investigating further as to why it is happening.

 

SL projects update week 41 / 1

Posted on by Inara Pey

Server Deploys

The main channel today received the same maintenance release made last week to BlueSteel (and which was used to help fix the LeTigre problems). speaking at the Server / Simulator User Group, Simon Linden described the deployment as, “A very minor change … from one of the RCs. It’s really not any functional change but was needed to go grid-wide before another one goes into RC.”

Following week 40’s issues with LeTigre, the three primary Release Candidate channels will be getting updated as follows:

  • Magnum, which has code to help sims run better on new hardware,  will be getting the same change that went to the main channel
  • BlueSteel and LeTigre will get the same update, which is described as “pretty minor but important”, being a back-end query optimisation designed to assist with database loads.

Currently, the prim accounting issue is still being worked on, which affecting scheduling what will be available in deployments. Passing comment on how deployment packages are put together, Simon explained that, as a rule, bug fixes tend to be put together as far as possible, prior to going to QA and then to an RC channel. He went on to say that what goes into other releases can be variable, “There’s a judgement call on how much gets bundled together, and a bunch of things go into the decision, like how overloaded the QA guys are, how many other things are trying to get to RC, risks of one part blocking it (like now), stuff like that”. In the meantime, LL are actively looking at ways to both prevent a recurrence of this problem and to improve the RC channel deployments as a whole.

SL Viewer

The promised new beta release (3.4.1.265642) reached the release point on Monday October 8th. This release sees tcmalloc disabled once more,  but otherwise appears to be the same code as the previous beta. It is intended to be a further stability testing / confirmation release, and as such will remain available for the next couple of days as Linden Lab gather data on its performance.Tcmalloc has been disabled, rather than removed, as it has apparently been useful in helping to trace issues within the viewer code, and LL wished to retain the ability to re-enable it in case they needed to re-enable it to help identify problems within the viewer in future.

Assuming this release proves stable, and assuming that plans outlined by Oz Linden have not undergone significant change, it should clear the way for the unblocking of various code merges that have been awaiting the stability / memory leak issue to be resolved. As previously reported, the precise order in which code merges will be made / released is unclear, but Linden Lab have a significant amount of updates waiting in the wings, including Steam support changes, Monty Linden’s HTTP library updates, Baker Linden’s Group Services project code, Apple OSX Mountain Lion support (including gatekeeper compatibility), and more.

Steam updates – one of the viewer merges waiting to be released

Under the original plans for the beta viewer, project viewer code was to start merging into the viewer with the 3.4.2 release code. As there was no OpenDev meeting on Monday 8th October, it is assumed this is still the case, however, the precise order of the merges is due for discussion this week within the Lab, and a clearer indication of the order may be available by the Thursday OpenDev meeting, and will be given in part 2 of this report, if that is the case.

Group Services Project

Due to the problems experienced with the leTigre deployment in week 40, Baker Linden’s Group Services code did not receive a proper deployment to a Release Channel. It is not due to be released this week, but should be in an RC deployment aimed at week 42, although as it has been bundled with the LeTigre deployment which had problems, this may be delayed further while the prim accounting error is looked into.

It is currently unclear as to whether the delay with the RC roll-out will influence when the Group Service viewer code (currently available in a dedicated project viewer) will be merged into the development  / beta branches of the SL viewer code; again more should be known on this following Thursday’s OpenDev meeting.

Materials Processing

Continues to progress, with little to report at this time. The feature set for the initial release still has yet to be published, and the wiki page for materials processing is due for further update. Concerns were raised over one statement relating to the use of colour, to whit:

“Color a solid color for the surface; not used if a Texture is also specified.”

The concern was that whereas it is currently possible to specify both a texture and a colour for a given object or object face, the wiki implies that under material processing, it will become either / or. However, this appears to be an error in the wiki, and both options will remain available.

Linkability Bug

As reported in my last update, and while not strictly a project, the bug which is currently allowing prims to be linked over distances greater than 54m has been investigated, and a fix is expected to be rolled-out to the RC channels for this in week 42.

 

SL projects update week 40 / 2

Posted on by Inara Pey

Server Deploys

As many are aware, there was a major error in this week’s LeTigre Release Channel deploy. Apparently, the root cause of the problem lay in the server-side prim account code, which Simon Linden describes as having “blown up” on the LeTigre RC channel. This resulted in a large number of items (including partial builds) being returned to people’s inventories as a result of regions being seen as “full”. The problem required a two-stage recovery:

  • LeTigre regions were rolled back to a state prior to the faulty deployment, and were then updated with the BlueSteel code also deployed on Wednesday October 3rd. This helped to determine the extent of the damage (a total of some 1200 regions)
  • The regions damaged by the land impact miscalculation were then restore to a state prior to the roll-out of the original faulty LeTigre code. These had to be restored manually, which took a considerable time

There is further post-mortem work going to to try and discover why this error did not reveal itself when the code deployed to LeTigre was being tested on Aditi, and whether there is anything specific to the regions impacted by the error which may have triggered it. Thought is now also being given to managing large scale region restorations, despite this being the first time there has been such a massive issue of this kind occurring across the grid.

Current RC plans for next week call for the same maintenance release to be made to all three RC channels, which Simon Linden describes as, “Mostly internal changes but [which] does include a minor update for the physics engine library … It’s almost all updating libraries … we’ve been using a fairly old set of compilers and such to make some of the development builds of the servers, and this brings us to more recent code.” Further details on the deploy should be available next week in the Second Life Server section of the Technology forum.

SL Viewer

As indicated in part one of this report earlier this week, problems have continued with the Beta viewer code and high crash rates. Work has been ongoing to try and locate the probable cause(s), some of which included the temporary return of tcmalloc. While not actually a cause of the crash issues, having tcmalloc disabled was affecting efforts to reproduce the problems. a beta release was made on the 3rd/4th October (3.4.1.265434), which is proving to be a lot more stable than previous versions, and which happens to have tcmalloc enabled.

The current plan is for a further beta release to be made, most likely on Monday 8th October, which should see tcmalloc turned off once more (if not removed). Should this also prove to be stable, the fixes it contains will be merged back into the development viewer code, and this will clear the way for clearing the backlog of code merges for both the beta and development viewers. It may also see a further 3.4.1 release version of the viewer being made.

Among the projects awaiting merging into the development and beta viewer code are:

  • The Steam support changes, which have been available within a development viewer stream, and which are described as “mostly cosmetic”. There is apparently a version of the viewer on Steam, but it is not available for general viewing / download, and is presumably there for testing purposes
  • Monty Linden’s HTTP library (texture fetch) code
  • Baker Linden’s Group Services project code
  • Apple OSX 10.8 Mountain Lion support work, including gatekeeper compatibility
  • Bug fixes and further regionalisation work.

Previous plans for these releases called for them to be made under the 3.4.2 code base. While this wasn’t discussed at the TPV/Dev meeting, one assumes this is still the case. However, speaking at the TPV Dev meeting on Friday October 5th, Oz Linden indicated that the order, etc., in which waiting merges will be cleared hasn’t been fully defined, and will be the subject of internal conversations next week at the Lab.

Avatar Baking Project

Bake fail: a familiar problem for many

There is still no major news on this project, although work is continuing both on the viewer and on the server code.

The plan remains to provide TPV developers with access to the viewer code at least 8 weeks ahead of any initial deployment of the server-side code to an Agni release channel. This is to allow TPVs time to merge the code into their viewers and participate in ongoing testing of the new service.There is a possibility that that viewer code will be available sufficiently well ahead of things in order for TPVs to be able to use it alongside the testing on Aditi (beta grid), depending on the status of the beta grid tests and how development of the viewer code progresses.

Please use the page numbers below left to continue reading this report

Pages: 1 2

SL projects update: week 40 / 1

Posted on by Inara Pey

Update 3rd October: The RC roll-outs were somewhat different than indicated by Simon. Magnum apparently received the back-end configuration work for new and future hardware, and BlueSteel received the maintenance update. The Group Services code & maintenance release was targeted at LeTigre, however a showstopper issue means that these regions are now in a state of flux. A comment from Oskar Linden reads: “LeTigre regions exhibited issues that necessitated a rollback. The regions were rolled forward and are on the same code that is on BlueSteel. Affected regions are getting a simstate rollback” . 

Server Deploys

There was no main channel roll-out this week, as expected, following the cancellation of last week’s RC channel deploys.

Wednesday 3rd October will see the three RC channel deploys originally planned for last week:

  • Back-end configuration work designed to help SL run better on new and future hardware – this should be deployed to BlueSteel
  • A maintenance release, which includes has Baker Linden’s Group Services project – this will be deployed to Magnum
  • The third is described by Simon Linden as being, “Very similar to what we have today, with a fix for some future compatibility coming down the pipe. It’s nothing really exciting, but required so things won’t break.” – this should be deployed to LeTigre.

The order of the releases is not clear at the time of writing, and is based on Simon Linden’s comments at the Simulator User Group.Confirmation / updates to the plan should be made available via the Server Deployment thread in the forums.

SL Viewer Releases

Things remain slow due to on-going crash / possible memory leak issues, as reported in my last mini-update.

Group Services Project

The Group Services project is an attempt to improve the management and editing of large SL groups by replacing the current UDP-based service (which has capacity issues with the size of group lists it can comfortably handle) with a new HTTP-based service. The project viewer for this is already available (for Windows, Linux and OSX.). It had been hoped that the viewer code might reach a 3.4.2 beta release during this week, however due to continuing issues with the current beta viewer code, this now seems unlikely.

The initial server code for this will be deployed to the Magnum RC channel, as mentioned above, on Wednesday 3rd October.

Materials Processing

The final feature set for the first release of materials processing is now more-or-less complete, and it should hopefully be announced nearer the time for beta testing. A number of regions on Aditi have been prepared for beta testing, and details on these will be released when the initial project viewer is ready for release.

Options to be used by Normal and Specular Maps, which will need to be provisioned in the Build floater

The initial feature set will include the ability to set parameters (rotation, offset, etc.), for normal and specular maps as well as diffuse (texture) maps. Oz originally hinted this was the aim a few weeks ago, although Geenz indicated it was only hoped these capabilities would be included, rather than it being definite at that time. This means that the build floater for the project viewer will be somewhat different from most people who build are familiar with, as it must include a number of additional options (see right). However, what is being considered is not a complete rebuild of the Build floater.

Commenting on this aspect of the work at the Content Creators’ Improvement Informal User Group, Oz Linden said, “We’ve got a strawman design in internal review… will have a version to look at soon, I think.”

The design has had input from a number of builders. Some of these are from within LL, some of them users, and will probably be left unchanged until further experience is gained in its use. Whether this means the viewer remaining unchanged between the beta programme and the release of materials processing on Agni, or whether changes are made between the beta and the release, remains to be seen. As Oz went on to comment, “It’s a very difficult problem, and we tried to meet three sometimes-conflicting goals: do what needs to be done; keep things familiar; make the things you have to change better in the process. I think we did pretty well…”

Obviously, TPVs will have access to the build UI code once it is available in LL’s accessible repositories, and they’ll doubtless look at the code in terms of how best to integrate it into the look and feel of their own viewers.

There are still no firm dates for the project in terms of beta commencement, etc., but Oz reiterated that the project will follow the familiar course, with an Aditi beta, followed by a release to one of the RC channels on Agni, prior to an eventual full roll-out.

Related Links

With thanks to JayR for the simulator UG meeting transcript.

SL projects update, September 26th

Posted on by Inara Pey
Update 28th September: Please also refer to an update post on some of the projects / news given here.

SL Viewer Status Updates

Linden Lab have been working hard on a range of viewer-related issues, notably crash rates and memory leaks, which have slowed the viewer update a release process up over the last few weeks. In terms of memory leaks, tcmalloc has been identified as the culprit, with Linden Lab deciding that dropping it is “probably a good idea”, according to Oz (tcmalloc has previously been implicated in crashes linked to the use of things like Microsoft’s Skydrive). There have also been an issues with LL’s statistics system which have meant that the viewer hasn’t necessarily been accurately tracked in terms of crash rates, etc.

Beta Releases

As it stands, LL hope to have the blocks on the various code merges removed during this week, which should see a rapid series of beta releases coming down the pipe. This work commenced with an initial 3.4.1 beta release (3.4.1.265134) emerging on September 24th. It will be followed by around three or so additional and rapid 3.4.1 build iterations aimed at confirming the viewer’s stability and at replacing various fixes which had previously been removed from the viewer code while trying to identify the causes for the viewer crashing / suffering memory leaks. It is expected that each of these iterations will be on the beta release build channel for a couple of days, prior to being replaced. Following these there will be a series of project updates, the first of which gatekeeper compliance project, which is also targeted for a 3.4.1 release build.

Project-related Releases

Once the stability of the beta viewer has been confirmed, it is anticipated that project-related code will be merged into the viewer, most likely starting with 3.4.2 builds. Among the releases planned for 3.4.2 is Monty Linden’s HTTP Library Services and Baker Linden’s Group Services code. These are currently targeted to reach the beta build channel in week 40 (week commencing Monday October 5th).

These releases will at some point include the Steam updates currently in a Development branch as well, which might in turn mean that Second Life could be ready to appear on Steam in the very near future, once these updates have reached a release version of the viewer.

Account creation prompt: heading for the beta viewer

Group Services Project

The Group Services project is an attempt to improve the management and editing of large SL groups by replacing the current UDP-based service (which has capacity issues with the size of group lists it can comfortably handle) with a new HTTP-based service. The project viewer for this is already available (for Windows, Linux and OSX.), however, as mentioned above, the current plan is to get this into the 3.4.2 build stream alongside the HTTP textures project, possibly in week 40.

Originally, the server code for this project was due to have been rolled to the RC channels during week 38, (week commencing September 17th), but the channel deploys were postponed after QA issues were found. As a consequence, the roll-out was due to take place on Wednesday 26th September, but has again been postponed.

There has been some confusion as to the aim of this project, with some people believing it is focused on fixing group chat issues such as  lag and chat failing to start. This is not the case at all; as stated above, the project is aimed at improving the management and editing of large groups (10K+) through the use of a new HTTP service.

HTTP Library Services

As indicated above, the first phase of this work, covering a new texture fetch service, should be appearing in a 3.4.2 beta release of the viewer in the near future.

HTTP Libraries project viewer: improved texture loading and rezzing

Please use the page numbers below left to continue reading this article

Pages: 1 2

Pathfinding: summary update

Posted on by Inara Pey

Linden Lab has been quietly working on pathfinding, clearing a range of bugs, updating the supporting documentation (some of which is still a work-in-progress) and providing more information for users aimed at clearing up misconceptions / misunderstandings. The following is a quick update on recent activities.

Lorca Linden’s FAQ

On September 18th, Lorca Linden posted a Pathfinding FAQ to the Second Life Server branch of the technology forum. While perhaps not the most high profile place in which to post the information, the FAQ nevertheless addresses a number of core issues related to pathfinding and makes a valuable read for anyone interested in using pathfinding or who wishes to understand more about pathfinding in general, rather than relying on hearsay.

One of the major misconceptions which is perhaps missing from the FAQ is that disabling pathfinding in a region will somehow “improve performance”.  In fact,m the only thing disabling pathfinding for a region does is to prevent pathfinding characters from operating; the underpinning Havok engine remains unchanged, and no Havok functionality related specifically to pathfinding is “turned off” in any way. So if there are no pathfinding characters being used within a region, disabling pathfinding will not improve the region’s performance, and any apparent improvement which may be noted is more than likely a placebo effect.

Pathfinding Tools In The Viewer

The latest release version of the SL viewer (3.3.4.264911) now includes the pathfinding tools, as do a number of TPVs, some of which I reported upon a while back, and which have since been joined by Firestorm (4.2.2.29837+); while Singularity (1.7.1+) also now provides some of the viewer-side tools / options associated with pathfinding.

Updates list of viewers monitored on this blog  which provide pathfinding support (click to enlarge)

Documentation-wise, work has been put in on cleaning up the existing wiki pages (although some are still somewhat out-of-date or difficult to follow as they presume a certain level of understanding). An updated list of pathfinding wiki documentation and related resources which I’ve previously published in the blog can be found in Related Links, below.

While there is still ongoing work in relation to a number of pathfinding bugs, the arrival of the pathfinding tools into the SL release viewer theoretically marks the point at which pathfinding might be considered “fully released” (as previously indicated by Lorca Linden). As such, it would be beneficial for Linden Lab to provide a formal blog post on the subject, including links to relevant resources such as those listed below in order to make the information more readily apparent to SL users, regardless as to how well (or otherwise) LL believe their blog is read.

Related Links