SL projects week 46/1: server, projects and general news

Server Deployments – Week 46

Deployments for the week are progressing as planned.

Main Channel

The main channel received the code which had been running on the Magnum RC channel as well as some updates.

This now means that the server-side Group Services code to improve the loading and editing of very large groups (10K+ members) is active right across the main grid – see the section on Group Services below for further information.

This roll-out restored functionality within the Estate Tools which allows region physics to be put in a condition of limited functionality, which is sometimes useful in dealing with issues and problems within a region. The capability was disabled around the time of the mesh roll-out, and has now been restored with this release. This caused some minor inconvenience on some regions (at least one), where the option has been enabled at some point, with the result that following their restart, objects within the region(s) were not functioning correctly. However, this was corrected without major incident.

Main channel release notes.

Release Candidates

Wednesday 13th November will see all three Release Candidate channel receive the same update package, including the BUG-166 update, which means that linksets with bounding boxes larger than 64m (in any dimension) are prevented from being rezzed if doing so will cause the object to collide with an avatar excluding the object owner.

Release notes:  Magnum, BlueSteel, LeTirgre.

Week 47 Deployments and Christmas Run-up

There will be no server roll-outs in week 47 (week commencing Monday 19th November) due to the forthcoming Thanksgiving weekend in the United States. There will be deployments between Thanksgiving and Christmas, but it currently looks as if these may be limited to a couple of weeks during that period according to Simon Linden (details on the likely number pending), and there will as usual be no releases over the Christmas / New Year period.

Interest List and Object Caching

Andrew Linden reports that the first phase of this work is drawing to a conclusion, and he is planning on having a possible demonstration of the capability on the beta (Aditi) grid on Thursday 15th November, most likely during the Bet Server User Group meeting.

The focus of this project is to optimise the data being sent to the viewer, information already cached on the viewer and the manner in which that data is used in order to ensure it is used more efficiently so that things rez both faster and in a more orderly manner than is currently the case.

Interest lists and object rezzing: initial srver code updates almost ready

Andrew reports that general performance on object rezzing should be improved, although the overall sorting element of the code (ensuring objects closer to an avatar’s camera position rez sooner than those further away) isn’t currently as rigorous as it could be. However, the server and viewer do now interact better, so less information is sent to the viewer relating to in-world items which are not visible within the current camera view for the viewer.

Commenting on demonstrating the capability when speaking at the Simulator User Group meeting on Tuesday 13th November, Andrew acknowledged that it may be difficult to achieve on Aditi, which is a relatively static environment (improvements will hopefully be more noticeable on regions where there is a lot of movement and activity); however, anyone interested in this work may want to try attending the Beta Server Group meeting on the 15th November, in case a demo is provided.

Currently, this represents the fist pass in Interest list improvements, and one which is liable to be heading to an RC channel in the near future – it does not require any specific viewer updates to work -, and Andrew expects to be building on this work in the future.

Group Services

As mentioned above, Baker Linden’s Group Services HTTP code is now available across the main grid. As there was some confusion evidenced on Plurk yesterday, here’s a quick re-cap on what this means:

  • The new code allows for improved loading of membership lists of very large groups, together with improved reliability in editing such groups (i.e. assigning roles, removing people, etc.), by the group moderators
  • The code requires a viewer update. At the time of writing, this is available with the official Second Life beta viewer (3.4.2.266708+), and the code will be filtering into the majority of popular TPVs as they update (and currently appears to be available in Zen (3.4.2.2+) and Niran’s Viewer (2.0.3.2262+) and Cool VL (1.26.4.38), all of which successfully loaded large group lists for me)
  • Until such time as the viewer-side code has been incorporated into TPVs, the “old” method of loading group lists into the viewer will still be available. However, viewers using the “old” method (a protocol referred to as UDP) will have group loading capped at 10K members. This means:
    • That for groups with 10K or fewer members, there will be no change regardless as to whether the viewer is using HTTP or UDP
    • But for groups large than 10K, viewers running the UDP code will be unable to load the group until such time as they have been updated to the new code
  • The code will not lead to any improvements in group chat reliability, and is not aimed at improving group chat.

Materials Processing and Avatar Baking

No news on either of these, beyond what has been previously reported in these pages. Materials processing has a test region on Aditi, but there is no timeline on when a project viewer is to be made available. For an overview of the initial capabilities for material processing, please see my project update here, and remember that the capabilities will be applicable to prims and mesh, but not directly to avatars or system layer clothing.

Avatar Baking is progressing, but without any significant update at this time, please refer to my last detailed update on this project for information.

Mesh Importer Fix

JIRA SH-3055 records a  problem with the official viewer’s mesh uploader which has been affecting people over the course of the year. The fix for this, released as a project viewer (3.4.2.266471, available for Windows, Linux and Mac OS) on November 5th, is still available for those experiencing uploader issues, although it is in the pipeline to be merged with the beta viewer now that crash issues seem to have been resolved. Bear in mind that – as Runitai states in his JIRA comment, the viewer is a pre-beat project version, and may include other bugs and problems. While reports on the JIRA seem to point to it being relatively stable, caution should still be taken if attempting to use it as a primary viewer.

Related Links

SL project news week 45/2: server news, viewer updates Steaming ahead, and surprises

Week 45 Deployments

The deployments schedule for this week (Tuesday 6th and Wednesday 7th November) went ahead as planned, namely:

  • Tuesday 6th: the Main Channel received get the code currently running on BlueSteel and LeTigre – release notes
  • Wedneday 7th:
    • Magnum received get fixes and updates to the code currently running there (including the Group Services code) – release notes
    • LeTigre and BlueSteel should get the next bug fix server in the pipeline, which includes the code currently on Magnum, and more – release notes (BS) and release notes (LT)

The main channel deployment now means that all regions are running on the same version of Havok with the exception of Magnum regions, which should be getting the update in week 46 (see below).

LeTigre and BlueSteel both have an additional “feature”: Linksets which have bounding boxes larger than 64m (in any dimension) are prevented from being rezzed if rezzing would cause the object to collide with an avatar excluding the object owner (BUG-166).

In addition, both LeTigre and BlueSteel include the following oft-requested bug fixes:

  • Script Time in the Statistics Bar now correctly shows 0ms when scripts are disabled in the sim (BUG-311)
  • Script error messages now include information about the object’s root prim, when certain operations fail due to the object’s pathfinding setting (PATHBUG-198).

A crash bug was also found in the Magnum code, and this has received attention, with the fix due to go out next week.

Week 46 Deployments

Things are gradually slowing down in preparation for the Thanksgiving code release freeze which will see a suspension of code deployments during the Thanksgiving week later in November. As it stands, the following roll-outs are planned for week 46 (week commencing Monday 12th November):

  • Main channel: should receive the code currently running on Magnum (including Baker Linden’s Group Services code – see later in this article)
  • Magnum: should receive the code currently running on BlueSteel and LeTigre, which will mean the entire main grid is now running the same version of Havok
  • BlueSteel, LeTigre and Magnum should also get the same additional updates (details yet to be specified).

Beta Viewer Update – Steaming Ahead with Project Code Merges

As indicated in Part 1 of this report, the crash issues impacting the beta viewer code have been resolved, and LL have been engaged in merging-up code into the beta and paving the way for the first of the 3.4.2 beta releases. These were always intended to have the code from some of the major SL projects which impacted the viewer, including Baker Linden’s Group Services code and Monty Linden’s HTTP texture fetch code.

The first 3.4.2 beta viewer was release on Thursday 8th November (3.4.2.266708), which includes a range of updates from the Lab as well as a number of contributed updates and improvements (see the release notes), although precisely which of the LL project elements are in the release isn’t obvious from the release notes themselves – the removal of JIRA numbers from the release note entries makes identifying updates, features and fixes that much harder, even though the JIRA items themselves are still open for public viewing.

One element that is clearly in the latest beta viewer release is the code for the steam link-up, as evidenced by the arrival of the new splash screen which I first reported on back in August 2012 – complete with a promotional piece for the Lab’s Pattern’s game.

Continue reading “SL project news week 45/2: server news, viewer updates Steaming ahead, and surprises” →

SL project news: week 44/4: server and viewer news

Quick Links

Server Deployments, Week 44

There was some confusion on the server channel deployments this week. As reported earlier this week, roll-outs were put back by a day due to issues within LL. This meant that the main channel deployment did not occur until Wednesday 31st October, and the RC channels on Thursday 1st November.

There was further confusion as the main channel had been scheduled to receive the code which had been on BlueSteel and Magnum, but it instead received code which had been on BlueSteel some two weeks ago, and caused some confusion on the Server Deployment forum discussion for the week.

Other than being 24 hours late, the RC releases were more-or-less as planned and previously documented:

  • BlueSteel received the Havok updates deployed to LeTigre, complete with the lHTTPRequest header “fix”
  • Magnum has the Group Services code from Baker Linden for handling the editing and managing of large in-world group lists. The code includes some fixes to problems found in the Snack RC deployment of the Group Services code (again, see part 2 of this report)
  • LeTigre received further updates for the Havok deployment, etc.

In addition, Magnum received a fix for the Estate Tools debug capability to disable collisions. When enabled, the option freezes all physical objects in a region in place and prevents avatars colliding with objects. As such, it is primarily a useful tool when trying to sort-out performance issues or locate and remove unwanted objects (although in the latter case, any TPV with Area Search included can achieve the same result. The option has been broken for some time, and the code deployed to Magnum should fix it.

Also on Magnum, Qie Niangao reported issues which may – or may not – be linked to previously encountered problems with llsensor(). However, it is not clear as to how widespread this might be.

Deployments for Week 45

As they currently stand, the plans for Week 45 (commencing Monday 5th November) are:

  • Tuesday 6th: the Main Channel should get the code currently running on Bluesteel and LeTigre  – so this will be the Havok updates, etc
  • Wedneday 7th:
    • Magnum should get fixes and updates to the code currently running there (including the Group Services code)
    • LeTigre and Bluesteel should get the next bug fix server in the pipeline, which includes the code currently on Magnum, and more.
  • Further details will be available as the release notes are issued

Threaded Region Crossings

This has been a project which has been going on for some time now, with Lindens working on it quietly in the background. As it hasn’t been openly mentioned in User Groups for some time, it had led some commentators to believe the project was no longer being worked on. However, as I’ve commented in these pages (and more recently quoted Baker Linden as commenting on it), the work has been progressing, and may now be nearing the time when it will see greater light of day. As a part of this, it looks like there might be a call for volunteers to participate in a “pile-on” test in the near future, specifically to test whether the new code is easing region crossing issues and generally leading to an improvement.

Avatar Baking

Avatar bake fail

Questions about server-side baking are a common occurrence at OpenDev and TPV meetings.

Currently (and as all too often indicated in these reports!) work is progressing on both sides of the equation within LL, but there is currently little of major impact to report. The emphasis has been on the viewer code (which will be used as the basis for the new Texture Compositing server the Lab will be implementing.

However, it remains that are still  no ETAs on the availability of either the viewer code or the server. The plan, however, remains that TPVs should receive around 2 months notice as to the availability of code for merging into test viewers. Given this, it remains highly unlikely that there will be visible progress with the project before the start of 2013.

Please use the page numbers below to continue reading

SL projects news week 44/2: server, group services, mesh and more

Quick Links

Server Updates

Deployments this week are being delayed some 24 hours, which means the main channel deployment will not take place until Wednesday 31st October, and the RX channels on Thursday November 1st. This has led to concerns that the main channel deployment / restarts may impact any Halloween events taking place across the grid on Wednesday 31st. Simon Linden promised to raise the concerns at an internal LL meeting taking place later on the 30th October, however, at the time of writing, the Grid Status page reports that the deployment will go ahead, starting at 05:00 SLT on the 31st.

In the meantime, the deployments for the week remain largely as previously indicated:

  • The main channel should receive the code currently on BlueSteel and Magnum. As mentioned above, the code has no externally visible changes but has some system level adjustments – release notes
  • LeTigre will receive a further update for the code currently running on it, which will include a number of bug fixes and pathfinding updates – release notes
  • On Friday 26th October, Simon Linden indicated to me that Magnum should be receiving the code planned for week 43 (the llSensor() problem has been fixed) which will include Baker Linden’s Group Services code currently on Snack, however, as of the Simulator User Group meeting on the 30th October, the final release notes for Magnum had yet to be published, so the update may still be in a state of flux
  • BlueSteel should get the same code as is on LeTigre.

Interest List

Andrew Linden has fixed a bug wherein some child prims in linkset fail to rez and he has carried out further work on performance issue he reported last time. This turned out to be an issue with the code which caused the simulator to send a full update of everything within view to the viewer each time an avatar within visual range moved. Understandably, on  crowded regions, this led to performance issues.

The code is in the process of being revised to ensure it only calls for “terse” updates to be sent to the viewer, which will help ensure more relevant information is sent to viewers when updating, which should reduce the performance hits.

Group Services

Baker Linden, speaking at the Simulator User Group on Tuesday 29th October, said that the server-side code for this project, which should improve the load times and editing abilities for very large group lists, seems to be working “moderately well” since the deployment to the Snack RC channel last week. However, some bugs have been found, Commenting on these, Baker explained that, “The group name, description, and other things don’t load right now on the Snack RCs.” However, the bugs have been investigated and fixes found, which should be merged into the code ahead of the planned deployments on Thursday November 1st. The fixes have also been tested on Aditi, where they’ve been found to induce a slight lag on group loading.

For the time being, testing this now code continues to require a dedicated SL project viewer (available for Windows, Mac and Linux), until such time as issues with the SL beta viewer code can be fixed and merges with viewer project code resumed / made available to TPV for integration.

Avatar Baking

Work is still progressing, Nyx Linden confirmed, talking at the content Creation User Group on Monday 28th October. How far down the road the work is, is unclear. The server-side of things will apparently be using the code being developed for the viewer, and it is this which is the focus of attention for the present, as has been the case for some time.In talking to TPV developers on the subject the last time the matter came up, Oz Linden confirmed LL’s plan is still to give TPVs “at least” 2 months (eight weeks) notice prior to anything being rolled out for testing, in order to give TPVs sufficient time to incorporate the code into project viewers of their own and assist with the overall testing.

SL Issues

Mesh Alpha Issue

Theresa Tennyson demonstrates the skinned mesh / alpha issue

Also during the Content Creation meeting, a problem with alpha textures as applied to worn skinned meshes was brought up. Theresa Tennyson demonstrated the issue during the meeting, which somewhat resembles the old invisiprim  / alpha issue.

Siana Gearz suggested two possible causes for the issue, “[The] first is that rigged mesh transparent surfaces appear to be drawn before prim transparent surfaces. [The] second issue is that the shader apparently writes depth for whole fragment, not just for relatively in transparent pixels.”

Nyx Linden requested a JIRA item be raised for the issue, again highlighting the problem with the recent JIRA changes, in that outside of those with access privileges to the new system, no-one could actually confirm if a JIRA had already been raised.

ETA 31st October: Seems a public JIRA on the matter is available – see MartinRJ’s comments which follow this article. Many thanks, Martin!

With thanks to Theresa Tennyson for the Simulator UG meeting transcript

SL project news: week 43/4: server updates

Week 43 Deployments

Main Channel

Tuesday October 23rd saw an update to the main channel which should have minimal impact on things, the changes having previously been on the Magnum RC channel. LL have been monitoring the deployment and have seen no adverse impacts.

BlueSteel RC

The BlueSteel RC channel received a further update to the current server maintenance project aimed at general stability improvement.

LeTigre

LeTigre is currently the focus of the server modernisation project, and as such received further updates to the deployment of Havok 2012.1. This means that mesh vehicles will continue to be unable to cross from LeTigre into regions running on other server channels for the time being.

As previously reported in part one of this week’s project news, LeTigre contains an updated cURL library which may cause problems for services using an external website for frequent data updates. The issue affects llHTTPRequest.

Essentially, the problem is that until now, the cURL library used by LL uses an explicit call to ensure data being returned from an external web service (such as information relating to the health status of breedable animals) is “fresh” data, rather than anything which may have been cached along the way. As such, specific functionality hasn’t previously been required within LSL to ensure this is the case.

However, The new cURL library deployed to LeTigre no longer attached the explicit call (technically a Pragma: no-cache header) to outgoing requests. This means that information being returned as a result of a call to an external service may in fact come from data cached along the way (such as from an intermediary server). Obviously, receiving “old” data would not be good for things like breedables, which could end up dying.

To overcome this, LL have added a new flag to the llHTTPRequest function to achieve the same result as used to be achieved via the old cURL library attaching the Pragma no cache “request”. In the meantime, anyone with breedables or other services which rely upon frequently updated data from an external web service have been advised to test their products on LeTigre.

Magnum RC

The Magnum RC channel was to have received a series of bug fixes, together with Baker Linden’s code for large Group Services. However, a showstopper issue was discovered with llSensor(). As a result, the release was rolled back and replaced with the BlueSteel deployment.

Also to have been included with this release is a new capability allowing the simulator to report information about script permissions granted to objects within a region. This capability requires an update to the viewer in order to be used (the code for which is currently held-up due to the beta viewer issues). Once the viewer code has been released, the new capability will allow users to list all items in a region to which they’ve granted permissions (such as items which have been granted animate permissions) and, if required, revoke them.

Please use the page numbers below to continue reading this article

SL project news week 43/1: Server updates and llHTTPRequest

SL Server Updates

A brief start-of week update with an important item on the LSL llHTTPRequest function.

Tuesday October 23rd saw an update to the main channel which should have minimal impact on things, “It’s a change that should make simulators run better on our new hardware,” Simon Linden explained at the Simulator User Group meeting on the 23rd.

Wednesday 24th, as previously indicated, should see the RC channels updated as follows:

  • Magnum should receive bug fixes together with Baker Linden’s Group Services project code (the viewer side of which is still blocked)
  • LeTigre should receive further updates for the new Havok code (which presumably include fixes for the crash loop situation Maestro reported in the Server Beta UG meeting (see above)
  • BlueSteel should receive, “Some more invisible changes that should help us deal with some problems like full disks that make servers very unhappy.”

Details on the deployments are, as usual, posted in the Second Life Server section of the Technology forum.

Third-party Web Caching and llHTTPREQUEST

Kelly Linden indicated that the updates to LeTigre in week 42 had some library updates. One of these updates was to the cURL library which changed its behavior specifically around caching.

Until now, outgoing requests on have a Pragma: no-cache header in them, because cURL added this to all requests, and thus ensured fresh data was returned. The change made to the cURL library on LeTigre means that this is no longer the case, so if the third-party web server has caching enabled, any outgoing llHTTPRequest might return previously cached results from the server, rather than fresh data.

Kelly noted that, “Systems that are most likely to be affected are those that frequently hit the exact same URL and expect the data to change. Maybe they are getting a counter or checking on something’s status, leading to problems with the likes of breedables dying, and so on.”

A workaround for this has been implemented for the llHTTPRequest in the form of a HTTP_CUSTOM_HEADER flag, which enables Pragma: no-cache to be specified manually.

While there have currently been no reports of this or similar happening so far, LL are continuing to discuss the potential impact. Further, it is expected that the , Kelly continued, “If anyone uses or develops systems they think *might* be affected please give them a try on LeTigre this week and let me know. Or if you know others that might, encourage them to test on LeTigre this week. Thanks.”

So, if you have created a product which uses an external web server for updates in the manner described above, etc. (or know anyone that does), you may want to test behaviours on a LeTigre region to see if your product is impacted prior to this change rolling out further across the grid.

SL Issues

Network Traffic and Sim Lag / Crash

This issue has been going on since the start of the month, and appears to affect regions with large numbers of people.

A bug report was raised on the issue (BUG-355), and has been imported by LL as a MAINT issue (MAINT-1682). However, there has been no feedback from LL as to the underlying cause, although investigations are continuing.