Tag: Server Updates

Pathfinding: Magnum RC roll-out, viewer tools and more

Posted on by Inara Pey

Pathfinding is drawing closer to a release across the main grid, and preparation work for the roll-out – which will constitute one of the biggest changes to SL – is underway on several fronts. This article is intended to be a high-level update on various elements of the project, gleaned from a variety of sources.

Magnum RC Roll-out

On Wednesday July 11th, the server-side pathfinding code was rolled-out to the Magnum Release Channel. There had been some predictions that this could lead to significant problems and issues as a result of the issues given within the release notes

Following the release, issues were experienced, notably with mesh vehicles, as reported on the forum thread discussing the releases for the week, and which have been rapidly responded to by Linden Lab personnel. there are still concerns around the roll-out and potential impact, and Linden Lab are continuing to monitor.

In discussing the RC toll-out at the TPV/Developer meeting held on Friday July 13th, Lorca Linden, the Associate Producer responsible for the project, commented: “OK, so pathfinding did go in RC on Magnum on Wednesday [July 11th]. As a whole, things are looking really, really good. We’re seeing very few crashes, the performance is working great we are seeing issues with some vehicles – definitely not all. That’s the only major hitch that we’re looking into, but as a whole the RC has been going quite smoothly.”

New Viewer Tools

As mentioned above, Lorca Linden (together with Stinson Linden and Prep Linden) from the pathfinding project attended the Friday TPV/Developer meeting on the 13th July, where they specifically discussed the viewer-side pathfinding tools. The tools are covered in detail in a new wiki page from Linden Lab, and may already be familiar to those who have been working on the pathfinding beta. They are currently in the Pathfinding Project Viewer, and will need to be incorporated in to TPVs as well. The wiki page provides comprehensive notes on the tools, complete with screen shots; the following in intended to provide a high-level summary and some background notes for those unfamiliar with the core elements of pathfinding, and to provide an overview of what this means for viewers going forward.

Navmesh and the Rebake Tool

For those not familiar with the term, navmesh is short for navigation mesh. This is a representation of a region’s geometry generated and used by the physics simulator to determine paths for pathfinding characters. The navmesh can be somewhat fluid in nature, depending upon what is going on in a region and what is being changed; a new path for a character, for example, will change a region’s navmesh. When this happens, the navmesh for the region needs to be updated, which can take some six hours  if left to update automatically.

To overcome this when pathfinding is rolled out, one of the new tools that will be appearing in the viewer will be a Rebake Region button. This will automatically appear at the bottom of the viewer window of all users within a region when the navmesh requires updating – regardless as to who may actually have altered the navmesh.

Rebake region button for navmesh updates (with thanks to Linden Lab)

Once baking has commenced, the button will fade on all viewers on which is displayed, indicating that an update is in progress (and preventing someone else from initiating a rebake). Once the rebake is complete and the navmesh is updated, the button will vanish from viewers.

Object Attribute Tools

By default, a navmesh treats all resident-made objects within the region in which it is active as obstacles that pathfinding characters must manoeuvre around. Obviously, this may not always be the case; there will be objects (e.g. stairs, ramps, sidewalks, floors, etc.), pathfinding characters need to traverse, climb, etc. This is achieved by altering the pathfinding atrributes associated with an object, and some of the new viewer tools are to allow this to be done and to also allow users to examine in-world objects to determine their status vis-a-vis pathfinding and how pathfinding characters will react to them.

These tools take the form of menu options and additional panels located in the Build and Object Profile floaters.

Dedicated Floaters

Also included in the tools are three dedicated pathfinding floater panels:

  • Linsket floater: designed to give advanced users and builders the ability to customize an area to achieve interesting effects with pathfinding-enabled characters
  • Character floater: designed primarily to help users to locate characters moving throughout a region and to identify the CPU cost of characters affecting the performance of a region
  • View / Test floater: intended for advanced users who are building pathfinding-enabled objects and characters.
Pathfinding characters floater (with thanks to Linden Lab)

Tool Status and TPV Integration

As mentioned above, these tools are all currently available in a Project Viewer. However, it is anticipated that they will be appearing in a Linden Lab beta viewer in “one to two weeks” (Lorca Linden). The tools themselves are regarded as feature complete by LL, and Lorca encouraged TPV developers at the meeting to consider integrating them into their viewers sooner rather than later.

Integrating the new tools in TPVs is liable to be in two parts:

  • An initial release containing the tools required for setting object attributes, etc.
  • A follow-up release incorporating the use of the Havok libraries Linden Lab is establishing and which will be made available under the new sub-licence arrangement.

The reasons for this are two-fold:

  • The attribute tools, etc., are vital for optimising pathfinding within regions and ensuring everything works correctly (e.g. to ensure pathfinding characters can climb the stairs they’re supposed to by climbing or walk along the prim sidewalk they are supposed to walk along, etc.
  • The Havok libraries are not yet available, although Oz hopes to have them in a position where he can talk in more detail to TPVs about them “pretty soon”, and while it is nice to be able to visualise the navmesh, etc., it is not quite such vital part of the pathfinding process.

Universal Tools

Alongside the viewer tools, pathfinding will see a set of universal tools rolled-out in console format. These will be available to region owners and estate managers and will allow them to change an entire class of object in a region to have certain pathfinding attributes once pathfinding goes live. Linden Lab are approaching this in terms of having all non-scripted objects set them to be static obstacles that pathfinding characters must manoeuvre around, while anything that is scripted is set to “dynamic”, as it is thought to be moving.

This obviously doesn’t fit all cases – vendor boards, for example are scripted, but they are hardly what can be termed “moving” objects. Indeed, it might be argued that the majority of scripted objects within a region are non-moving, and therefore should have their pathfinding attribute set to “static”. However, LL feel they have no way of easily differentiating between a non-moving scripted object and a moving scripted object, and thus feel that setting all scripted objects to “dynamic” is the better option and allowing the attribute to then be modified through the viewer where necessary, as setting them to “static” could result in a worse overall behaviour case within a region.

Other Tools and Items

Alongside the above, Linden Lab have previously indicated that they will be making the following available as pathfinding rolls-out:

  • A set of script templates used for the creatures found in the Wilderness areas
  • A script for a “master rezzer system”.

The latter is a means by which region performance and the number of pathfinding characters rezzed in the region can be monitored, and which will reduce the number of pathfinding characters within a region in response to the region’s performance / number of avatars within the region.

Potential Timescales for Roll-out

During the TPV/Developer’s meeting, Lorca outlined some potential dates for pathfinding. note that these are currently potential, and shouldn’t be taken as tablets of stone:

  • The pathfinding tools should be available in one to two weeks in a beta viewer
  • The server-side release is dependent upon how well (or otherwise) the current release to the Magnum RC progresses, and may potentially come within the next two-to-four weeks, but certainly no sooner than two weeks.

Again these are not confirmed dates, and may well change in the next couple of weeks – particularly if major issues are found with the Magnum RC roll-out.

Related Links

User Experience Tools: initial roll-out to Magnum RC

Posted on by Inara Pey

Today sees the first phase in rolling-out the new User Experience tools to the Main grid. As noted in the official release notes, the tools have been rolled-out to the Magnum Release Channel.

The new tools were previewed back in March in a rare SL blog post, which I covered at the time it was released. Today’s Magnum release adds three new LSL functions for User Experience:

  • llAttachToAvatarTemp (integer attach_point) — Follows the same convention as llAttachToAvatar, with the exception that the object will not create inventory for the user, and will disappear on detach, or disconnect. It should be noted that when an object is attached temporarily, a user cannot ‘take’ or ‘drop’ the object that is attached to them. Additionally, if this function is used with experience permissions, the user is ‘automatically’ made the owner of the object. If you use this function without the experience permission, the target MUST be the owner of the object for it to attach properly
  • llTeleportAgent (key agent_uuid, string lm_name, vector landing_point, vector look_at_point) — Teleport Agent allows the script to teleport an agent to either a local coordinate in the current region or to a remote location specified by a landmark. If the destination is local, the lm_name argument is a blank string. The landing point and look at point are respected for this call. If the destination is remote, the object must have a landmark in its inventory with the teleport agent script. lm_name refers to the name of the landmark in inventory
  • llTeleportAgentGlobalCoords (key avatar, vector global_coordinates, vector region_coordinates, vector look_at) — Teleports an agent to region_coordinates within a region at the specified global_coordinates. The agent lands facing the position defined by look_at local coordinates. A region’s global coordinates can be retrieved using llRequestSimulatorData(region_name, DATA_SIM_POS).

This initial roll-out does not include the expanded Experience Permissions System, as Oskar Linden points out in a forum post on this week’s server releases. Instead, the new functions work with the current runtime permissions system (specifically PERMISSION_TELEPORT), although plans are in-hand to roll-out the new permissions systme at some point in the future.

No details have yet been released on the Professional Creators Programme that was mentioned in the original preview blog post, but if you are interested in learning more about the tools, the Advanced Creator Tools Notification Group is still open to membership, and you are encouraged to join the Group.

Oskar notes that LL will be actively monitoring the forum thread announcing the roll-out, and anyone encountering issues with the new functions is encouraged to post feedback in the thread, cross-referencing any relevant JIRA they raise.

Given the functions are now on the Magnum RC, and people are being encouraged to provide feedback, this roll-out would appear to move the User Experience tools outside of the associated Closed Beta programme.

For ease of reference, here’s the video LL released with the original preview announcement:

Related Links

Linden Lab roll-out Region Idling

Posted on by Inara Pey

Simon Linden has posted to the Server topic area of the technology forum about a new SL server feature, “region idling” which commences today, Wednesday 16th May.

This will see those regions that do not have any avatars either in them or camming into them to lower their frame rates and script processing, thus reducing their load on their host CPU. This should in turn improve the performance of the other regions running on the same hardware.

The idling itself should be entirely transparent to users, with the region immediately returning to “full speed” should anyone enter the region or cam into it. However, Simon does warn of a possible caveat to the transparency:

We expect this feature to be totally transparent to users. Residents will not see or be on regions that are idling. Scripts, however, may observe the effect if they are using the llGetRegionTimeDilation() function, and may require fixing.

There are some additional points to note with this capability, which are addressed in an FAQ also posted to the forum.  These include:

  • Regions are not turned off or shut down. They merely run at a slower frame rate when nobody is there. They will appear exactly the same way as before in search, the world map and other Second Life features
  • Region idling cannot be manually disabled for a region
  • Any scripts that use LSL network functions will suspend region idling for a short period of time to allow them to function normally.  This will allow scripts that connect to outside services via email, http and xmlrpc to run as expected.

The roll-out of the function will commence with the Blue Steel release channel, and will then be progressively rolled-out to the rest of the grid in the coming weeks. Anyone suspecting region idling is having an adverse effect on their region is requested to file a JIRA (no specific project given), providing clear information on the problem and the exact times it happened.

Related Links

Mesh: the roll-out

Posted on by Inara Pey

[POSTED 8:22PM PDT Aug 22, 2011] Rolling restarts for regions on the main Second Life server channel will be performed on Tuesday, August 23rd at approximately 5:00am PDT. Please make sure to save all builds and refrain from making purchases during this time.

Mesh: transformative? (w/thanks for Claudia222 Jewell)

With this simple notice, mesh commenced its roll-out to the remaining portion of the Main gird that has yet to see it (approximately 70%), as per the server release notes.So, assuming all goes well, the entire Main grid will be mesh enabled from today.

This isn’t the end of the mesh project, but really the beginning; thing’s are going to suddenly change overnight – it’s going to take time for mesh to make its presence felt, and as Linden Lab themselves admit, there will be further updates, revisions and improvements ahead.

As a result of this, we can doubtless expect the Beta “Viewer 3” to go to full release status shortly (if not today as well), which brings with it mesh support with some added features. Please see the update at the end of this article.

For those wishing to see mesh, issues still remain: the code for rendering mesh objects has yet to be incorporated into a number of third-party 2.x Viewers, and it unclear as to whether the code can be backported into the 1.x code.

In the meantime – a video from rockerfaerie featuring some of Claudia222 Jewell’s remarkable mesh art.

Updates

At 1300 SLT Nyx Linden confirmed the following:

  • All channels of the Main grid now support mesh except the BlueSteel RC channel (which one assumes will get mesh during the RC channel restarts on Wednesday 24th August)
  • The Default official Viewer is now Viewer 3 (currently 3.0.0.238864)

More on Mesh

Update 25th August

The “additional feature” referred to in this article was the ability to disable mesh rendering in a region. However, in commenting on a JIRA raised in relation to the capability, Charlar Linden stated:

Charlar Linden updated SH-2341:Status: Expected Behavior  (was: Open)   Resolution: Not Applicable – thanks – this flag would have no effect, and is being removed soon.

So it would appear the option to disable mesh is not intended to be a part of the Viewer.

Of channels and restarts

Posted on by Inara Pey

Once upon a time server roll-outs for Second Life were handled in what, on the surface, would seem a fairly straightforward manner:

  • New code would be tested on the Beta Grid, with users reporting any bugs or issues to LL for fixing
  • When considered relatively stable, the code would be rolled out on a limited basis to the Main Grid (affecting around 20% of the grid in total) for further “testing”; if major problems were found, the limited roll-out (or “pilot”), would be rolled back
  • If considered stable, the code would be rolled out to the remaining 80% of the grid, generally around 24 hours after the pilot.

The system wasn’t flawless; the complexity of the server code meant that many small (and one would guess conflicting) updates would be “rolled-up” into a single release, often with unpredictable results, despite testing on the Beta Grid. This would result in what I call the “tidal effect”: a change would be rolled out as a pilot, then rolled back for fixing, then rolled out before being rolled back for further fixing, and then rolled out once more, then rolled out again to the entire Main Grid. Sometimes even then, it would go through one more rollback / rollout.

As we’re all only too aware, this approach meant fairly large and constant upheavals for just about everyone concerned, and the cause of much gnashing of teeth and dark mutterings towards Linden Lab.

To try and minimise the overall impact of server code updates and roll-outs, Linden Lab switched over to a “channel” system. Under this system, server code is operated across four channels: the Release Channel, with the latest “release version” of the server code (and supposedly the most stable), and three “Release Candidate” channels, code-named Blue Steel, Magnum and Le Tigre.

Each of the RC channels comprises about 10% of the total Main Grid, and is used to roll-out a “beta” of a specific server code package. This might be a series of bug fixes (e.g. specific SVC JIRA fixes), it might be a general maintenance release (e.g. security updates, etc.), or it may be related to a specific, on-going project (such as display Names, the “Fast Assets” project, the “Inventory Capabilities” project, and so on). Broadly speaking, specific projects tend to be rolled out through specific channels (The Inventory Capabilities project tends to rollout via Blue Steel, for example, as do changes related to the forthcoming arrival of Mesh) – although this is not a hard and fast rule. General maintenance releases, on the other hand, are distributed between all three channels, depending on which has the capacity at the time a release package is ready for beta testing.

So, at any one point in time, some 30% of the grid is hosting what is effectively “beta” software, but in very discrete “chunks”, so to speak, confined to known sets of simulators. The releases themselves are also smaller and more easily managed / identifiable, making everything that much easier to manage and, in theory at least, making issues that much easier to identify and correct.

Broadly speaking, this is how it works:

  • An update (be it bug fixes or whatever) is readied for release as a “beta”. If it is related to a specific project it may be targeted at a specific RC channel (Blue Steel, Magnum or le Tigre)
  •  On the Wednesday of each week, the Release Candidates for each channel are rolled out to their respective 10% of the Grid; if a specific channel doesn’t have a candidate waiting, this obviously, nothing is rolled out
  • Over the course of the next week, the candidate’s performance and impact on the Main Grid is monitored (and the channel servers may be subjected to numerous restarts. If the candidate proves particularly problematic, it may even be rolled back
  • If the candidate appears to be stable after 6 days, then it is (together with any candidates from the other two channels) rolled out to the entire Main Grid the following Tuesday
  • The cycle then repeats with the next RC in the channel dropping into its assigned servers on Wednesday.

If a specific RC causes problems, then the cycle for a specific channel may be broken for a week while the issue is worked on (for example, if a candidate on Le Tigre, say, proves that it is not ready for release as scheduled on a Wednesday, it will be “held over” for a week and made ready for release the next Wednesday).

There is one other channel worth mentioning that doesn’t get a lot of publicity: the “Snack” channel which handles releases related to (among other things) Mono-2 updates and various script monitoring tools. These are known to behave unpredictably, and so are initially rolled out to a very limited number of sims for testing. I understand that once tested, the fixes then go on for wider testing via (usually) Magnum prior to a full rollout.

The benefits of this system are obvious: if there is a major problem with a Release Candidate, it will only affect 10% of the Main Grid (rather than 20% with the old system); the releases are less complex, making it easier for the root cause of specific problems to be identified and corrected. Overall, the process means that there is less widespread upheaval across the Main Grid than tended to be the case with the old, larger-scale releases. There are many examples of these advantages; when a recent change impacted breedable horses, for example, it only affected a small percentage of horses on the grid (only those present on servers running the specific release channel software).

Of course, there are what appear to be downsides to the new system: the release channels (particularly, it would seem, Le Tigre), can be in a state of flux when problems do occur; and the weekly rollouts, with their need for sim restarts, on both Tuesday and Wednesday has been the topic of many a complaint. A minor irritant is the pop-up that comes up when moving between sims running different server releases, be they a release channel or the “full” release – it would be nice if these could be turned off by those who have no interest in what software is being run on a given simulator, just as other pop-ups can be user-disabled through the Viewer.

But, these grumbles aside, it has to be said the new system works. While it does cause a degree of pain for those “stuck” on simulators running one of the release channels, the vast majority of the grid has seen far less upset and upheaval when things have gone wrong. Certainly, the “tidal effect” of gird-wide rollouts/rollbacks has become largely a thing of the past, and while the rolling restarts associated with Tuesdays and Wednesdays might seem inconvenient when they begin, the truth is they’re probably less so than they were under the old system.

Server 1.38 rolls outs

Posted on by Inara Pey

Following the recent announcement on the subject, sever release 1.38 has started arriving, and with it the first steps in script management, together with other goodies.

Some 20% of the grid will be used as a pilot environment during the roll-out, with the rest of the grid receiving the update (assuming the pilot goes well) by the 6th April.

For scripters / builders, there are a series of new LSL commands than mean that (a) a lot of us are going to be busy for a while and (b) overall script counts / memory loading for certain functions should be reduced as the new functions are adopted. Initially, the ability to incorporate the new functions (for those that use them) will be limited to those working on 1.38 server sims (obviously) – so many of us are going to have to wait until after the roll-out has been completed before we start determining what needs to be changed.

As far as the script management tools are concerned, these are only available for those using Viewer 2.0; the functionality is not being back-ported to Viewer 1.2x (unless third-party developers opt to do so).

Ciaran Laval currently gives an overview of the new About Land script information tabs, and if it is not old hat by the time my home sim receives 1.38, I’ll likely have a few words on the information displays as well – unless, of course, Linden Lab keep to Jack’s stated promise and provide the necessary Information themselves. In this regard, and even allowing for the current roll-out being a pilot, I was somewhat surprised that nothing on the situation was posted in the LL blog (although a brief note did eventually appear in the Grid Status links).

Reaction to the new server code has so far been good –  those in the pilot are reporting good stability and overall improvements in sim performances. Doubtless, part of the latter is down to 1.38 fixing the Mono rezzing / start-up bug that would cause massive lag spikes.

Providing LL communicate the new script management tools and their limitations clearly, and estate owners can communicate matters to their tenants as well, 1.38 will hopefully be a boon to SL overall.