Tag: Interest List

SL projects update week 18 (2): server releases

Posted on by Inara Pey

Deployments for Week 18

The week 18 deployments make for interesting reading.

SLS Main Channel

On Tuesday April 30th, the Main channel was rolled back to release 13.04.05.273580, as a result of a widespread performance issue.

Release Candidate Channels

  • BlueSteel and LeTigre: both of these channels should remain on the Experience Keys project, but will also be reverting some changes, due to the same performance issue which is affecting the Main channel – release notes
  • Magnum: should remain on the same server maintenance project as week 17.  This project brings some new minor features to LSL, and fixes some crash modes.  This update fixes the grid performance issue, and fixes an issue in which llDialog() messages sent to the object owner could be incorrectly throttled – release notes.

The performance issue which caused the Main channel to see the re-deployment of an older release was described by Simon Linden as being related to problems with regions locating their neighbours. “The performance problem was really showing up between any one region trying to locate another on the grid … the system was actually working, but too close to the cliff for comfort.” The re-deployment means that the new LSL AO capabilities can no longer be compiled / run on any Main, BlueSteel or LeTigre regions, until the supporting code is rolled-out once more.

There is still no further detail on the Experience Keys project and whether this may / may not be more than a deployment of the Advanced Creation Tools permission system.

Interest List Update

Andrew Linden has been working on fixing a bug related to Meeroos (but which I’ve seen affecting other animals as well), which he describes as, “If you turn your camera away from a crowd of Meeroos, wait several seconds, then turn back around… the Meeroos will be updated, but not quite in the right order. So sometimes you’ll see a head move to the new position, then a fraction of a second later the rest of the body.  So I have a theoretical fix that doesn’t crash the simulator (anymore).” The fix in question has yet to be tested and QA’d, so there is no time frame for any release.

“Missing” Prims

While talking about the interest list work, Andrew answered a question on missing prims / linksets, again acknowledging it to be a viewer-side issue, before going on to say:

We think maybe it is fixed in a new viewer. But this new viewer I mention happens to be very crashy, so we haven’t opened up the source code for it yet nor have we submitted it to our QA team since they’ll just crash …  This is the viewer that goes with our new interest list changes which I mentioned a few weeks ago and people were wondering when the code would be put up on a public repo.

"Missing" prims - viewer-side fix possibly on the way?
“Missing” prims – viewer-side fix possibly on the way?

So a viewer-side fix, along with viewer-side interest list updates, looks to be somewhere on the horizon.

Region Crossings

There have been a number of reports of region crossings worsening again after seeing a significant improvement with the release of the fix for BUG-1814. A common issue has been avatars becoming “snagged” at a region boundary while the vehicle they were travelling in continuing on its way, sometimes being returned to their Lost and Found folder from a location two or three regions beyond where the avatar became stuck. Both prim/sculpt and mesh vehicles are affected when the problem occurs, and it is an issue which had been encountered prior to the widespread deployment of the BUG-1814 fix.

Getting "snagged" at a region crossing while my aircraft flew on was a problem I encountered several times over Blake Sea early in April. The problem has again manifested itself to many, and I've again encountered it while flying my mesh Spitfire Mark IX
Getting “snagged” at a region crossing while my aircraft flew on was a problem I encountered several times over Blake Sea early in April. The problem has again manifested itself to many, and I’ve again encountered it while flying my mesh Spitfire Mark IX

I’d actually encountered the problem on April 4th during a series of region crossing tests, but the problem no longer appeared to be occurring by the middle of the month.

The issue of region crossings was raised at the Simulator Meeting on Tuesday May 30th, but the discussion was dominated by the problems being encountered by one particular type of train. In commenting more generally on region crossings, Andrew Linden said, “I agree that region crossing on vehicles needs more work. I can’t promise that I’ll be working on that as soon as I’m done with this interest list project, but I’ll try to bring it up in the next ‘what do we work on next’ brainstorm that we have.”

Interest list and object caching update

Posted on by Inara Pey

In June 2012, Linden Lab announced the Shining Project, aimed at implementing a range of improvements over time designed to improve Second Life’s overall improvements. Part of this work was defined as “Object Caching and Interest Lists”, which the blog post described as “Intending to provide faster drawing of objects that are most likely to be redrawn later in a resident’s session or in the next session.”

Andrew Linden has been developing this project over the last several months, as reported in an ongoing basis in this blog. So far his work has primarily been focused on one side of the equation – the interest list code itself, which has been primarily server-side work.

However, with the most recent updates, Andrew has started looking at the “object caching” side of the equation, with changes made to how the server defines “cacheable objects” held by the viewer. With the next set of upcoming updates, this work is set to expand – and, for the first time within the project – see changes made to the viewer as well.

Current Issues

Before examining the forthcoming updates Andrew has been working on, a quick update on issues people have noticed in relation to interest lists changes.

"Missing" prims - not caused by the interest list work, but possibly exacerbated by it
“Missing” prims – not caused by the interest list work, but possibly exacerbated by it

“Missing” Prims: There has been a marked increase in prims in linksets failing to render, requiring a right-click (as one solution) to make them visible. As already reported in my week 15 project updates, this is not an interest list problem per se, although Andrew views the increase in occurrences of the problem as possibly being related to his changes in how the server defines “cacheable objects”. His investigations are continuing.

Objects out-of-view failing to produce sounds: If an object is initially out of your field-of-view, it is not sent to the viewer until such time as it does enter your field of view, and so sounds from it are not played. Andrew has not had time to look into this, but notes that it is “on his radar”

Finally, and while not a technical issue per se, Andrew reports that some users are still unhappy with the way the server sends updates from objects – particularly those close to your camera  – when they pass out of your field-of-view. While not entirely sure how this can be addressed, Andrew is considering what he calls a “keyhole idea”, wherein instead of just sending updates for objects in the camera’s current field of view  – the camera frustrum  – the server might send updates for all objects within a sphere of a set radius around the camera, as well is its frustrum. However, the problem with this idea is how big a radius should the sphere be? As Andrew notes, “Whatever it is set to probably wouldn’t work for everyone,” so he is still considering options.

Forthcoming Updates

Speaking at the Simulator User Group Meeting on Tuesday April 9th, Andrew revealed that the next Release Candidate submission of his interest list work is now ready. This comprises three elements:

  • Optimisations to reduce CPU cost of sending interest list-related data
  • Improved streaming through re-balancing the bandwidth used by the various UDP message categories
  • New “hints” the server will accept from the viewer to help it build the interest list for the viewer.

This work is aimed in part at making more intelligent use of the information on a region the viewer may already have locally cached from a previous visit, and it relies on both the initial information the viewer sends to the server on connecting to it, and what Andrew Linden calls “cache probes” sent by the server to the viewer. The approach, as explained by Andrew, basically works like this:

  • On connecting to a region for the first time (or after cache has been cleared), the viewer can “tell” the server it doesn’t have any information on the region, resulting in the server sending the viewer all the data it needs
  • If the viewer already has data relating to the region, the server send out a series of “cache probes” (lists of object identifiers – known as their local_id – and their versions), which the viewer then checks against its list of cached identifiers (local_id) and their versions
    • If the cached version number of an object’s local_id matches that sent by the server, the viewer doesn’t send any reply to the server, and the object is rendered directly from cache
    • If the cached version number does not match that sent by the server for a given local_id, it indicates the object has changed since the last time it was cached by the viewer, and a “cache miss” message is sent to the server in reply, which prompts the server to send the updated data, which the viewer then uses when rendering.

    Continue reading “Interest list and object caching update” →

SL projects update week 12 (2): server, interest list, region crossings, particles

Posted on by Inara Pey

Server Deployments – Week 12

The planned server deployed went ahead as scheduled and as follows:

  • Second Life Server (Main channel): on Tuesday 19th March, the SLS channel received the server maintenance package which had been re-deployed to Magnum in week 11. As with the Magnum re-deploy, it excludes the fix for VWR-786 while LL go “back to the drawing board” to try to correct issues. However, it does include the following two fixes:
    • BUG-1612: region Owners and estate managers finding they are unable to teleport back to their region after disabling direct teleports to the region
    • SVC-8019: region visibility delays following region restarts.
  • Release Candidate channels: on Wednesday 20th March, the three RC channels received the following updates:
  • BlueSteel and LeTigre: received the same updates as deployed to the SLS channel on Tuesday March 19th, but otherwise retain the same updates received in week 11 – release notes (BlueSteel)
  • Magnum: should receive further updates to Andrew Linden’s interest list work, as per the release notes.  Specific interest list bug fixes included with this update comprise:
    • Updates for objects that are out of view are delayed for a maximum of 5 seconds, at which point they will be sent (mitigates BUG-1779)
    • Fix for “No object updates from vehicles after some region crossings” (BUG-1814) – see below
    • Fix for “Agent appears in incorrect position to other agents after being moved by a sim teleporter” (BUG-1795).

Interest List Updates

Vehicle Crossings

The Magnum RC channel received a potential fix for the vehicle crossing issues (BUG-1814) which have been experienced by many of late, and which appears to have been a major step in the right direction, going on feedback posted to the deployment thread on the forums.

Essentially, the problem lay in the fact that the recently deployed interest list code has only been updating the viewer with information relating to objects within the camera’s field-of-view. If an object moves out of the field-of-view for more than a few seconds, then updates cease until such time as it re-enters the field-of-view.

Where this becomes a problem when crossing a region in a vehicle, is that when transferring between regions, the camera position is effectively extrapolated first, and any vehicle the avatar is attached to is created “behind” the camera by the simulator just entered. Thus, to the interest list code, the vehicle is “outside” the camera’s field-of-view, and so updates on it are no longer sent to the viewer, and problems result.

The updated interest list code deployed to Magnum ensures updates relating to any object an avatar is sitting on are always maintained (or “subscribed to”, to use the official term) by the interest list code, regardless as to where the vehicle appears to be relative to the camera position. Thus, updates continue either side of a region crossing, allowing control of the vehicle to be maintained.

I took time out to test the code fix, with the aid of Erick Gregan, on Thursday March 21st and we agreed that for aircraft at least, it looked good – neither of us encoutered problems.

Flying a Spitfire XI by Erick Gregan over the four Magnum Sandbox regions, I encountered no isues with sim crossings
Flying a Spitfire XI by Erick Gregan over the four Magnum Sandbox regions, I encountered no issues with sim crossings

Those wishing to test the updated code across region boundaries with aircraft or ground vehicles can do so by visiting the Four Magnum Sandbox regions (accessible to members of the Beta Tester SL group – which is free to join). The update itself will hopefully be deployed to the entire grid in the week 13 deployments.

Object Updates

The lack of updates being received from objects outside of the camera’s field-of-view have resulted in other issues as well, which are also fixed in the Magnum release.

For example, if you are looking at a cube which is changing colour between red and blue every few seconds, and turn your camera so that it is outside your field of view and wait for it to turn blue before camming back, the cube will briefly still appear to be red in your view before suddenly changing to blue. This is due to the fact that when the cube is outside your camera’s field of view, the updates associated with it are no longer sent to your viewer until such time as it re-enters your field of view.

With the updates deployed to Magnum, updates for objects outside your field of view are once again sent to the viewer – but at  a very low rate (around one update every 5 seconds). While this uses a little more bandwidth then sending no updates at all, it also means that objects in a state of change behave more predictably when camming away from and back to them (BUG-1779).

Cacheable Objects

The Magnum update also implements the first set of changes to the re-use of cacheable objects held by the viewer in order to speed up rezzing / rendering. These changes take two forms: a change in what can be cached in the viewer, and a change the order in which things are rendered.

Up until now, any object or item containing a script was considered non-cacheable by the viewer, whether or not the object itself has changed at all.  So if the object disappears (because you teleport away from the region, for example), then all data relating to that object is discarded by the viewer, and has to be re-obtained from the simulator on re-entering the region, which slows rendering.

Continue reading “SL projects update week 12 (2): server, interest list, region crossings, particles” →

SL project updates: week 11 (2): Server, SSB testing and HTTP news

Posted on by Inara Pey

Update Match 18th: The “mesh only” HTTP test regions, referred to as “coming soon” in the HTTP section of this report are now online on Aditi, and are called MeshTest2 (DRTSIM-203), MeshTest2A (DRTSIM-203A) and MeshTest2H (DRTSIM-203H), respectively. In addition, the sandbox areas (unrestricted access) are also now available as Sandbox HTTP (DRTSIM-203), Sandbox HTTP A (DRTSIM-203A) and Sandbox HTTP H (DRTSIM-203H).

Server Deployments

The planned deployments for the week went ahead as scheduled. Namely:

  • On Tuesday March 12th, the Second Life Server (SLS) Main channel received Baker Linden’s large (as in complex) object rezzing project, designed to improve region performance when rezzing large objects – release notes
  • On Wednesday March 13th, the BlueSteel and LeTigre Release Candidate (RC) channels received a server maintenance package, intended to fix a common crash mode – release notes
  • Also on Wednesday March 13th, the Magnum RC channel received an update to the server maintenance package deployed in week 10, with further improvements / fixes. These included the removal of the fix for VWR-786, which rather than correctly fixing the known issue (IMs to friends not respecting their privacy settings) resulted in all IMs to non friends returning the “User is not online” message, regardless as to whether the recipient was online or not. Release notes for the package are on the SL wiki. The Lab, according to Maestro Linden, is going to have to, “Go back to the drawing board,” to resolve this problem.

As I’ve mentioned in past reports, the aim of this “large object” rezzing project from Baker Linden is to improve how complex objects (those which have a lot of scripts and / or a large file size), with the focus of the work moving the parsing of object files onto a background thread in order to prevent the simulator being choked and performance spiking when such objects are rezzed. As such, the work applies to both in-world objects and attachments, with mesh potentially being a primary beneficiary of the changes. The thought is that the new code may also help frustrate griefers, as the simulator FPS should be better when multiple scripted objects are being rezzed.

SL Viewer

The SL beta and development viewer were both updated on March 14th. The beta viewer moved to release 3.5.0.271843, with updates primarily aimed at CHUI, as anticipated. The development viewer moved to release 3.5.1.271846, and remains broadly in sync with the beta version of the viewer.

Vivox  Updates

Alongside the FMODex (sound system) updates which will be forthcoming after the formal release of CHUI as mentioned in part 1 of this report, Vivox (the SL Voice service) is also due to be updated to version 4.5. Like FMODex, this is unlikely to happen until after CHUI has reached a formal release., but once implemented, this should result in an improvement in Voice quality.

Mesh Deformer

There is still no news on this. Both the issues relating to avatar shapes and weighting are still awaiting internal resources at LL. As such, there is no timeline as to when any movement might be seen on this project.

Interest List – Issues and Further Updates

More interest list updates on Aditi and likely heading for a Release Candidate channel in week 12.
More interest list updates on Aditi and likely heading for a Release Candidate channel in week 12.

The following fixes, related to the interest list code, should be in an RC deployment in week 12 (week commencing Monday March 18th):

  • A fix issue where you ‘lose track’ of a vehicle after a region crossing
  • A fix for BUG-1795 (“Agent appears in incorrect position to other agents after being moved by a sim teleporter”), which should see an end to avatars still appearing in view after they have used a teleport system
  • A fix for the issue where object moving off-camera would suddenly ‘snap’ into place when you turned your camera so they were in your field-of-view. Whereas up until now, the new interest list code has not sent any updates for such objects, the fix to be deployed in week 12 will once again allow updates to be sent to the viewer, but at a much lower rate than before the interest list code was originally deployed.

The code with these updates is currently available for testing on two regions on Aditi: Solariam (rated: Adult) and Tischeriidae (rated: Moderate). This code also includes updates to improve object rendering (particularly with the viewer set to low bandwidths) and to object cacheing as well, as noted in the first part of this report, and those wishing to do so are encouraged to do so, although testing any improvements to vehicle regions crossings might be contingent upon being Adult verified.

Server-side Baking (SSB) Pile-on / Load Test

The second SSB pile-on load test took place on Aditi on Thursday March 14th, immediately following the Server Beta User Group meeting. The test was undertaken using the latest version of the Sunshine project viewer and appeared to be broadly satisfactory, involving both users and a fair few LL personnel – Nyx (x2, as he had his alt there), Simon, Maestro, Monty, Dan, Don, Log, and others. As with the original test, this took place on two regions – one running the current baking system, the other running the new service. Participants were asked to change outfits using the current service, so that the project team could get some baseline / comparison stats, before everyone moved over to the “new” service on the adjoining region.

Continue reading “SL project updates: week 11 (2): Server, SSB testing and HTTP news” →

SL project updates week 11 (1): server deployments, viewer, and interest list updates

Posted on by Inara Pey

Server Deployments

Second Life Server (Main channel)

On Tuesday March 12th, the Main channel received Baker Linden’s large object rezzing project which had been deployed to BlueSteel and LeTigre in week 10. This project is designed to improve simulator handling of “large” (as a file size / complexity, rather than physical object size) so that the simulator does not stall / choke when handling one or more such objects. makes sim performance smoother while objects are being rezzed. Further details on the project are in my week 10 update, and the server release notes are available in the SL wiki.

Release Candidate Channels

On Wednesday March 13th, the Release Candidate channels should be updated as follows:

  • BlueSteel and LeTigre: both of these channels should be receiving the same server maintenance package, intended to fix a common crash mode – release notes
  • Magnum should receive an update to the server maintenance package it received in week 10, with further improvements / fixes. These include the removal of the fix for VWR-786, which rather than correctly fixing the known issue (IMs to friends do not respect their privacy settings) resulted in all IMs to non friends returning the “User is not online” message, regardless as to whether the recipient was online or not. Release notes for the package are on the SL wiki

SL Viewer

There has been no major viewer movement since the last update in week 10.

CHUI, the Communication Hub User Interface

CHUI looks set to be merged-in to the Snowglobe code, with Oz Linden stating he was hoping to start on this on Monday 11th March. There are concerns as to how LL’s ongoing work with the viewer might impact TPVs going forward. As it is, CHUI is liable to remain in the SL beta viewer for a while (and there is expected to be one more CHUI release into beta, as perviously noted).

Materials Project

Work is continuing on clearing the current issues within the viewer code, with a further push of the non-public viewer expected this week, which may resolve some of the problems.

The Potential LL Roadmap for Viewer Releases

While things are always in a state of flux, the potential order of viewer releases from LL’s perspective is currently veering towards:

  • Materials is now unlikely to “be seen” until after the code has merged with the CHUI code  – this follow-on from the SSB code currently undergoing a merge with CHUI and the move to merge CHUI with the Snowglobe code mentioned above
  • This does not mean that a materials project viewer will not appear prior to CHUI reaching the SL release viewer; rather it means that when a materials project viewer appears, it is likely to have CHUI incorporated into it
  • That said, materials will likely only arrive in the release version of the SL viewer after CHUI has been formally released and (most likely) SSB has been deployed
  • Other updates – FMODex is currently awaiting CHUI as well, but has no clear release date; the same is true of the Mac Cocoa project. Currently, it appears as if these are unlikely to reach mainstream release until after CHUI has been formally released.

Most TPVs are currently focusing on the Server-side Baking (SSB) integration, as this has a significantly greater impact on viewers in terms of the impact on  users than CHUI (although the latter is by far the most complex update as it involves a lot of code refactoring as well as CHUI updates). As such, it is likely to be a while before CHUI starts appearing in the majority of third-party viewers (although Kokua has already merged with it, and now has the CHUI code in the beta branch of its code).

Merchant Outbox Project Viewer

As reported over the weekend, Linden Lab has re-issued the Merchant Outbox project viewer, updated to the 3.4.4 viewer code, but which does not incorporate CHUI. This release is purely to assist merchants who are encountering issues in migrating to Direct Delivery now that the initial retirement of Magic Boxes has been announced.

Those who have / are encountering problems in migrating to Direct Delivery can obtain the viewer from the Alternate Viewers wiki page.

This project viewer will be withdrawn at some point in the future, and will not impact other viewer releases.

Server-side Baking

A further reminder that there will be a further SSB pile-on / load test on Thursday March 14th, following-on, as with the last test, from the Server Beta meeting on Aditi. For wishing to participate:

  • The test is liable to be in much the same format as the first test
  • Those participating should be running the latest version of the official SSB project viewer (3.4.5.271419)
  • Participants should have a number of outfits of system clothing, preferably with multiple layers, which they can swap between during the course of the test. Library outfits are acceptable, but LL are keen for people to use their own outfits to add greater weight to the tests
  • Clearing the viewer cache prior to the test is suggested, but not an absolute requirement.
The first SSB pile-on / load test (image courtesy of Latif Khalifa
The first SSB pile-on / load test (image courtesy of Latif Khalifa

The project also seems to be going through a further informal name change: originally referred to as “Server-side Baked Texture Generation & Storage”, the project has generally been shortened to “Server-side Baking”, but is now tending to be referred to as “Server-side Appearance project”. I’ll be continuing to refer to it as “Server-side Baking” or “SSB” for ease of reference.

Continue reading “SL project updates week 11 (1): server deployments, viewer, and interest list updates” →

SL project update: week 9 (2): servers, HTTP, SSB

Posted on by Inara Pey

Server Deployments – week 9

Server Life Server (Main) Channel

As noted in part 1 of this report, the Second Life Server (SLS / Main channel) received the server maintenance project which was on all three RC channels in week 8, comprising miscellaneous bug fixes, and the improved region restart notification. Deployed on Tuesday 26th February, the roll-out was followed by a number of issues being reported via the deployment forum thread.

Issues have included:

  • Parcels / regions vanishing from search for a period of time following the restart (an issue WolfBaginski Bearsfoot gave some thought to in commenting on part 1 of this report)
  • Further region crossing issues, with loss of control followed by “broken” camera positioning after recovery. This issue has been reported at some of the Simulator and Server Beta meetings previously, and Eric Gregan has produced a video demonstrating the problem as it occurs on some aircraft, although he’s also come up with a means of avoiding the issue which may or may not work for people.

  • Problems with Dwarfins attachments.

Release Candidate Channels

On Wednesday February 27th, all three Release Candidate channels received a new server maintenance package which includes a fix for a crash mode (see the release notes (BlueSteel)). No significant issues have been reported for this deployment.

SL Viewer

The Communications Hub User Interface finally made its debut in both the SL development and SL beta servers, arriving as release 3.5.1.270826 for the development viewer on Tuesday 26th February, and 3.5.0.270825 for the beta viewer on Wednesday 27th. I’ve provided a brief introduction to CHUI, and there is also a video from Torley Linden.

CHUI is liable to remain in beta for a “Good long run”, to quote Oz Linden. Hopefully, this may mean that materials processing will be the next significant viewer update to arrive as a project viewer.

Interest List

The recent updates to Andrew Linden’s interest list work are apparently won’t reach a RC channel in week 10, but should see deployment in week 11 (week commencing Monday 11th March). This work includes fixing issues with avatar appearance teleporters which use llSetRegionPos(), as well as correcting problems where objects which change appearance behind the camera ‘snapping’ into place when you rotate the camera back.

Server-side Baking

On Friday March 1st, Linden Lab unexpectedly released an updated version of the SSB project viewer (release 3.4.5.271118), which includes their approach to overcoming the problems of the Z-offset capability common to many TPVs being broken as result of the SSB code. The new approach, which I was able to outline when it “launched”, introduces an additional shape slider into the Edit Shape floater; as such, the ease-of-use of the approach, particularly for those who may swap between different avatar shapes (e.g. “normal” and petite) is very questionable.

The new "Hover" option in the Edit Shape panel for adjusting avatar height offsets in the Sunshine project viewer. Not the most elegant solution
The new “Hover” option in the Edit Shape panel for adjusting avatar height offsets in the Sunshine project viewer. Not the most elegant solution

Continue reading “SL project update: week 9 (2): servers, HTTP, SSB” →