Tag: Server Updates

SL project news: week 3 (1): Servers, Materials, Baking and more

Posted on by Inara Pey

Update 19th January: As is being reported in the deployment thread in the forums, further issues are arising with search, and also problems with scripted vehicles (as Wolf also nones in the comments for this article. Those using regions running on Magnum are advised to keep an eye on the deployment thread for news on issues as they are investigated by other users / feedback from the Lab.

Update 18th January: An issue has ben reported with Magnum regions no longer being listed in the main search floater, although shops, etc, within a Magnum region are still listed. The problem does not extend to the world map search. Maestro Linden has filed a bug report. A rolling restart of all LeTigre and BlueSteel regions commenced at 10:00 SLT due to instablility issues being reported with the regions. The restart will comprise a roll-back of server code on BlueSteel and LeTigre to 12.12.18.268345.

Deployments for Week 3

After the issues encountered last, week, there was no main channel release on Tuesday 15th January, 2013.

Changes also occurred with the RC channel deployments. As fixes for the threaded region crossing problems were not ready for release, this was removed from the schedule and replaced by a maint-server release. Then, a network hardware error forced the RC deployments to be postponed until Thursday 17th January.

The roll-outs duly took place on the 17th, and comprise:

BlueSteel and LeTigre received the interest list improvement project. Originally scheduled for Magnum in week 2, but was held up due to a last-minute bugs, this  update should reduce the bandwidth usage of viewers due to object updates, and should improve simulator performance, especially in sims with many connected avatars – release notes here (BlueSteel).

This release received an additional bug fix, related to seated avatars temporarily vanishing from other avatars’ view when  crossing between regions. This fix may have led to the server release notes as reported and wiki pages for BlueSteel and LeTigre slipping out of sync with the actual code release (this has now been corrected subsequent to my pointing it out to Maestro Linden).

Magnum received a new server maintenance project, which mostly contains bug fixes, together with the following:

  • Server-side code to support “neck” and “center” attach points, which didn’t get rolled out when these were added to the viewer (but which now also require a minor change to the viewer, which is forthcoming)
  • An update to improve the effectiveness of estate bans (i.e. preventing banned avatars returning)
  • An update for builders, which covers SVC-7996 and as a part of STORM-68 (ensuring all created items have the default permissions set within the viewer). This particular fix ensures that scripts created using the New Script button in the build floater inherit the permissions set within the viewer, rather than server-defined defaults
  • An update to how abandoned mainland is handled, as the automated buy-back option for abandoned land is being eliminated (see: Abandoned Land in the Knowledge Base)
  • Release notes here.

Deployments for Week 4 (Commencing Monday 21st January)

There is no major news on releases for week 4, other than the threaded region crossing code now has fixes for the last-minute issues which prevented its roll-out in week 2, so this should be on one of the RC channels for deployment on Wednesday 23rd January.

SL Viewer

The release version of the SL viewer rolled to the 3.4.4 codebase on Tuesday January 15th, with the release of 3.4.4.268864. See the release notes for details of updates and fixes.The development viewer is rapidly progressing through a series of releases, with 3.4.6.269073 released on January 15th, followed by 3.4.6.269108 on January 17th.

After a series of development version releases, the CHUI project viewer also moved to the 3.4.4 codebase with the release of version 3.4.4.268981, also on January 15th. The Development version of CHUI continues to keep pace with releases from viewer-development.

Server-side Baking

Work is continuing on avatar baking (otherwise know as server-side baking or Project Sunshine), with Nyx Linden reporting at the Content Creation User Group on Monday 14th January that the team has, “Been a bit heads-down focusing on some bug-stomping.”

Concern is growing that, since the initial release of the Sunshine Project viewer a month ago, there have been no further updates to the viewer-side code, despite extensive feedback from TPVs, which  – in the words of one developer – is not conducive to enabling everyone to adopt the latest code and get ready for the baking service within anticipated time frames (i.e. mid-to-late February).

Nyx has acknowledged the problem, and hopes that the Sunshine code will be merged-up to the latest viewer development code soon, together with verification that nothing has been broken as a result. In the meantime, and away from the SL viewer itself, Henri Beauchamp reported that he has server-side baking working in the experimental branch of his Cool VL viewer.

A further concern with the project is the manner in which JIRAs are being handled since last year’s changes to the system. Raised issues are effectively being cloned, with one version sitting internally to LL and “hidden” from general viewing, with the cloned version available for public viewing and update. This has resulted in worries that cooperation between TPVs and LL in resolving issues related to the Sunshine project could be hampered if the cloned public JIRA are not properly updated whenever LL update the internal versions of the same issues. Nyx Linden has promised to work to ensure that all cloned public JIRA are kept up-to-date.

Materials Processing

The server-side code for this is “inching towards” a point where it will be ready for deployment to an RC channel. When this will be is unclear, but it is likely to be ahead of any project viewer emerging from the Lab.

Concern is still being raised over the need to run the viewer in deferred mode in order to see materials processing in action once it has been rolled-out onto the grid. Some of this concern may be down to the confusing way in which deferred mode is labelled in the Graphics tab of the viewer preferences (“Lighting and Shadows”), with people thinking they must have shadows running in order to be in deferred mode (i.e. with the Shadows drop-down set to either Sun/Mon or Sun/Moon + Projectors). However, the viewer is running in deferred mode even with the drop-down set to None.

Continue reading “SL project news: week 3 (1): Servers, Materials, Baking and more” →

SL project news: week 2 (3): server, mesh and materials

Posted on by Inara Pey

Server Deployments – week 2

As noted in the update to part 2 of this week’s report, the planned deployment of two new releases to the RC channels didn’t go as anticipated. Originally, it had been intended that BlueSteel and LeTigre would received the new threaded region crossing code while Magnum would receive Andrew Linden’s interest list code improvements.

Interest List Deployment Cancellation

The interest list deployment was cancelled after the 11th hour discovery of some bugs with the code. Speaking at the Server Beta meeting on Thursday 10th January, Maestro Linden described the main issues as – ironically – being connected with region crossings, and with object updates.

In the first, anyone crossing between regions several times in a vehicle would experience all of their non-rigged attachments disappearing from their world view, with the viewer itself eventually physically detaching them. Not only did this cause confusion as to what was happening with attachments for those experiencing the issue, it also resulted in some avatars ending up naked following a relog.

Interest listcode: bugs led to deployment cancellation on Wednesday 9th January
Interest list code: bugs led to deployment cancellation on Wednesday 9th January

The second problem was slightly more complicated, if potentially more rare. In it, if User A had an object on the ground and User B looked at then turned their camera away such that the object was no longer on their screen, User A could then wear the object as an attachment and teleport away; however, when User B subsequently turned their camera back to where the object had been, they would still see it on the ground despite the fact it had been taken away. What happened if User A (now wearing the object) teleported back to User B wasn’t actually tested.

As a result of both of these issues and the cancellation of the interest list deployment, Magnum received the same region crossing package as intended for BlueSteel and LeTigre.

Region Crossing Code Issues

However, the bad news did not end there, as Maestro Linden explained, “After a few hours, we saw that the [region/sim] crash rate was way too high.” As a result, the threaded region crossing code was disabled via a configuration change to the servers without the need to rollback the release. Once this had happened, region crash rates returned to “normal” levels. In all the new region crossing code was active for around five hours before being disabled once more.

Analysis of the crash rate revealed it to be linked to avatars crossing to / from heavily scripted regions. While the new code was extensively tested on Aditi, the regions there were not excessively loaded with scripts during testing, and so the problem did not manifest. However, subsequent testing with the test regions running heavy script loads did result in them also crashing, confirming the problem.

At the time of writing, Kelly Linden believes he has a fix for the issue; if so, it should hopefully find its way to the RC channels in week 3, commencing Monday 14th January.

Continue reading “SL project news: week 2 (3): server, mesh and materials” →

SL project news: week 2 (2): Server deployments and network issues

Posted on by Inara Pey

Update:  The Interest list release to Magnum has been postponed due to a last-minute bug being found. Magnum has instead received the same code  as deployed to Bluesteel and LeTigre, containing the threaded region crossing update.

Server Deployments for Week 2

Server deployment recommenced this week, with a main channel roll-out on Tuesday 8th January, which saw that channel get the same code as released to the RC channels just before the Christmas / New Year break and as reported here.

Wednesday January 9th will see two major deployments to the RC channels. LeTigre and BlueSteel should be receiving a package which includes threaded region crossing code while Magnum gets Andrew Linden’s work on Interest Lists.

The threaded region crossing code should help improve simulator performance when avatars are region crossing to and from a region. However, and as previously noted, while the code did give clear improvements when crossing between regions on foot, the results were less positive when using vehicles during a recent test. Whether this has been improved as a result of those tests remains to be seen.

Release notes are available for both for LeTigre and Bluesteel, and the forum discussion thread is available for feedback / questions.

Interest List Deployment

The Interest List work should reduce the bandwidth usage of viewers due to object updates, and should improve simulator performance, especially in sims with many connected avatars and which is busy in terms of object updates.

This is the first phase of work planned around interest lists and object caching as a part of the Shining project. Andrew’s work has primarily been focused on improving the manner in which object updates are handled, etc., in order to provide improved performance. However, even with this initial phase of the work, there should be some improvements in the actual order in which objects appear in world (those closer to your camera position appearing prior to items further away – although there is far more work to be done in this regards before the project is finished. 

As it stands, there are already some updates to the work in the offing, as Andrew indicated at the Simulator User Group meeting on the 8th January, saying, “I’ve got some interest list work that didn’t quite make it into the Magnum RC … this work is almost ready for testing, so I’ll be trying to get it into a maintenance branch or something for a follow-up release.”

Interest list updates should be deployed to the Magnum RC on Weds 9th Jan.
Interest list updates should be deployed to the Magnum RC on Weds 9th Jan.

The new interest list code will see the final removal of the “legacy cloud” layer (at around the 170-200m mark). While this has been disabled viewer-side since around the introduction of Viewer 2, the data relating to the legacy cloud layer has still been sent out by the servers, allowing viewers which still incorporate the necessary code to render the clouds in-world. However, the server-side code relating to legacy clouds has been removed from the interest list code, so viewers will no longer be able to render the clouds.

The Magnum release notes are available here.

Continue reading “SL project news: week 2 (2): Server deployments and network issues” →

SL project news: week 1, 2013: forthcoming RC releases, viewer, and new work

Posted on by Inara Pey

RC Deployments for Week 2

The Lab is still getting back up to speed following the Christmas / New Year break, so expect further information to be forthcoming on Main and RC releases for week 2, 2013 via the Server topic of the Technology Forum.

However, as it stands, there are two projects which it is hoped will reach RC channel release in the week commencing Monday 7th January, 2013. These are Caleb Linden’s threaded region crossing code and Andre Linden’s interest list code.

As I’ve previously reported, the threaded region crossing code was subjected to a pile-on test on Aditi towards the end of 2012. The results were, on the whole, a little disappointing for those taking part – although expectations may have been set a little high. While there were some improvements noted – particularly when travelling between regions on foot and with a heavy script load – overall, there were sill issues with crossing between regions in vehicles (particularly ground vehicles).

Airbourne antics: vehicles still exhibited region crossing issues duing the threaded region pile-on test in December 2012
Airbourne antics: vehicles still exhibited region crossing issues duing the threaded region pile-on test in December 2012

Issues arising from the pile-on test are still being looked at, and Caleb repeated his request that anyone noting specific issues should raise a JIRA directed for his attention. For those wishing to try out the code, the GC Test regions are still available on Aditi.

The Interest List code is still subject to receiving an OK from the Lab’s QA team. There will doubtless be an update on this – and on the planned RC releases in general – at the Simulator User Group meeting on Tuesday 8th January, 2013.

SL Viewer News

Not a lot to report on here at present. The Beta viewer reached the 3.4.4 code base Just before Christmas 2012, (3.4.4.268497, December 20, 2012). There may be a rendering issue which may require addressing and might lead to a slight delay in releases; apparently, not all tests are giving the same results, so LL are still investigating the matter. Work is continuing to update the GPU tables for the viewer; further cards have been added to the table, and several blanket entries have been removed (such as all unrecognised nVidia cards being detected as nVidia Ion GPUs).

As reported over Christmas, CHUI rolled through a number of rapid releases in its development version, and the main project version rolling to 3.4.3.268587 on December 22nd. Both the development and project versions of the viewer are on the 3.4.3 codebase, and the most recent development release was made on January 4th (3.4.3.268703). Both versions are available from the Alternative Viewers download page.

While the core of the Mac version of the viewer is built using OSX 10.7 (with Xcode 4.3.3), work is progressing in moving the viewer to OSX 10.8 Mountain Lion, which is expected to happen “very soon” according to Oz Linden, although no date is available as to when.

New Pathfinding Capability

VoidPointer Linden is working on a new flag for pathfinding characters. STAY_WITHIN_PARCEL is designed so that when set, pathfinding characters will only set goal points during wander, evade, pursuit, etc that are within the parcel they get created in. If the parcel is a non-regular shape, it is still possible a character will cross between it and neighbouring parcels (unless the navmesh is cut through the use of an exclusion volume), but goal points will only be set within the originating parcel. The code is still in development, and so the constraints on where a character can wander when it comes to irregular parcel shapes, but VoidPointer is not making any promises on this.

He's completely batty! - Voidpoint Linde at the Server Beta UG meeting, Jan 3rd, 2013
He’s completely batty! – Voidpointer Linden’s avatar at the Server Beta UG meeting, Jan 3rd, 2013

There is no stated delivery time for this new feature, other than it is currently being worked on.

Server Object Rezzing Code

Baker Linden has been looking to improve how objects with large file sizes are handled by the simulator software when being rezzed. He describes the work thus, “What I’ve been working on is hopefully significantly decreasing lag spikes when rezzing large, complex objects. Large does not necessarily imply size, but size of the files being read. When an object is rezzing, we have to parse the object / mesh files and create our in-world objects with that data.”

Until now, reading and parsing of any files related to objects which require rezzing has been on the main thread. When several such objects requiring rezzing at the same time, the simulator stalls. Baker has been moving the reading / parsing operation to a background thread in the expectation that rezzing multiple “large” (again, in terms of file size, not the size of the object itself) objects will not choke the simulator.

The key point about this work is that it is specifically aimed at preventing the simulator processes from choking and a region stalling when there are a number of large object files being read / parsed, not at actually “speeding up” the physical rezzing process. As such, it is unlikely that objects will appear any faster in people’s in-world view as a result of this work. However, what it does mean is that the simulator code will be better able to handle rezzing multiple “large file” objects without the attendant region lagging which can occur as a result of the simulator being unable to process messages from viewers and other simulators, etc.

Information Sources

  • Opensource Developer meeting, Wednesday 2nd January, 2013
  • Beta Server meeting, Thursday 3rd January, 2013.

Related Links

SL project news week 51/2

Posted on by Inara Pey

Server Deployments

The RC channel deployment took place on Wednesday 19th December as planned, with no major hiccups or issues apparently being felt / reported. This release saw all three RC channels receive the same package, which is the Magnum deployment from week 50, with a single additional bug fix.

The release notes (for Magnum, but applicable to all three RC channels) can be found here, and the forum discussion thread is still open should anyone have comments (allowing to the start of the holiday period & delays in any replies from LL).

SL Viewer Update

The last beta release of 2012 arrived on Thursday December 20th with the release of 3.4.4.268479 – which as the name suggests, uses the 3.4.4 code from viewer-development. Chief among the updates in this release include:

  • Moving viewer crash logger and SLPlugin from the Carbon API to Mac OSX Cocoa for the Mac viewer
  • Fixes for fontconfig crash-on-start issues experienced with Linux
  • Revisions for pathfinding options, including additional of tear-off option for Pathfinding menu when opened from Build menu; introduction of Region Rebake option to Pathfinding menu
  • Fixes for 3.4.2.266708 release issue of flexiprims stopping flexing after teleports
  • Further fixes for llTargetOmega() issues

The full set of release notes can be found here.

Forthcoming Viewer Graphics Enhancements

Alongside of the upcoming materials processing capabilities of normal and specular maps, the SL viewer will be receiving further rendering improvements in 2013, comprising:

  • Gamma correction capabilities
  • Improvements to the “Shiny” capabilities in the viewer (Build floater > Texture tab >Shiny button) when running in deferred mode, to make shine appear more as it does in the real world, with the level of shine based on the amount of reflected light / angle from which a shiny object is viewed from relative to the light source
  • In-world scene reflections on shiny surfaces.
Gamma correction: from Exodus to the SL viewer in 2013
Gamma correction: from Exodus to the SL viewer in 2013

Gamma correction is a process by which lighting on a screen can be adjusted to make it look more natural to the human eye by adjusting the RGB channels to give a more natural-light contrast. It can also be used to produce interesting photographic effects. Exodus viewer already has gamma correction, which can be adjusted via the Visual Settings floater (above).

A simple, if extreme, example of gamma correction: both images were taken under the same lighting conditions
A simple, if extreme, example of gamma correction used for a photographic effect: both images were taken under the same deferred lighting conditions, the image of the left with “normal” (default) gamma correction in the viewer, the one on the right taken with all three channels increased in value to produce an image with a much deeper contrast

As with materials processing, all three of the new capabilities are being introduced to the SL viewer via the Exodus team, lead by Geenz Spad, who indicated that in the case of gamma correction, the new capabilities will be somewhat more capable as a process within the SL viewer than is currently the case with the Exodus viewer.

As with material processing itself, which is still progressing, as I reported last time, there are no time frames as to when each of these new capabilities will be available within the viewer.

Threaded Region Crossing Code Test

Wednesday 19th December saw a “pile-on” test take place on Aditi for the new threaded region crossing code. The test has been called by Caleb Linden with a view to trying-out region crossing under a range of tests, including avatars with heavy scripted loads, repeated crossings by vehicles, via teleporting, etc.

In all, six regions were / are available for tests: GC Test 9, 10, 15 and 16 forming a block of four running in “threaded” mode, and GC Test 2 and 8 running in “unthreaded” mode, but otherwise the same server code.

Turn-out for the tests wasn’t particularly great (around 15 people together in the test region at any one time), with a range of ground and air vehicles being used to try-out the crossings and compare them directly with crossing between the two unthreaded regions (and testing between threaded / unthreaded).

The overall consensus from those at the tests were that crossings between threaded regions were somewhat improved – but the overall improvements were not that spectacular. At times recovery from loss of control of a vehicle appeared to be faster – a matter of one or two seconds – but at other times, things seemed to be the same, with vehicles ploughing through the ground / flying into the air for 6-7 seconds at a time before recovering, giving pretty similar results to those encountered crossing between threaded / unthreaded and between unthreaded regions.

Crossing between threaded regions in a vehicle: some improvements, but not a lot
Crossing between threaded regions in a vehicle: some improvements, but not a lot

Crossing between regions on foot, even under an excessive HUD / attachment script load (373 scripts accounting for 2.87800ms cpu time) and with vehicles zinging back and forth, did appear to me to be somewhat improved, with little or no “rubber banding” or walking off into the sunset, but a recovery of control after just a few paces.

Post-test, Caleb has requested those participating in the test file JIRA on any specific issues they encountered, and include “Region Crossing” in the description / subject heading for the report.

With thanks to Darien Cauldwell for information on the viewer enhancements discussed at the Opensource Dev meeting.

SL project news: week 51

Posted on by Inara Pey

Server Deployments week 51

Tuesday 18th December saw the Main Release channel get the maint-server release as deployed to Magnum in week 50, as Maestro Linden indicated might be the case during the Server Beta meeting on the 13th December. There have been some reports of teleport issues following the deployment, but these are thought to be a network-side issue, rather than a problem with the server code itself.

Release notes.

All three RC channels should be receiving the same package on Wednesday 19th December, 2012 which is also the Magnum deployment from week 50, with a single additional bug fix.

Release notes (for Magnum, but applicable to all three RC channels)

Forum thread for the deployments.

This is the last set of server-side deployments for 2012.

A low-key Simulator User Group meeting on 18th December, the last for 2012
A low-key Simulator User Group meeting on 18th December, the last for 2012

SL Viewer Updates

The release version of the official SL viewer rolled to the 3.4.3 code on Tuesday 18th December, with the release of 3.4.3.268262. Among a range of updates, this release include the major GPU table updates seen in recent beta viewer releases, the additional graphics settings and the tiling fix for high-resolution snapshot (MAINT-638). This release also sees the return of JIRA numbers to the release notes, matching the recent beta viewer release notes update.

The new "intermediary" graphics settings intended to better represent the capabilities of different GPU classes
The new “intermediary” graphics settings intended to better represent the default SL capabilities of different GPU classes

The beta viewer should be moving to the 3.4.4 code with the next release, although it is not clear whether this will be before or after the holiday period.

Interest List Update

Andrew Linden has been tweaking and optimising his Interest list code. The first phase of the new interest list code is aimed at reducing the amount of information being sent to the viewer by the server. Speaking at the Simulator User Group meeting on the 18th December, Andrew commented, “Last week I managed to find an optimisation opportunity in my Interest List project, which shaves a few hundred usec off of some cases where there are a lot of Full Object Updates. So this would translate to better server FPS when in large groups of avatars or when in the presence of stuff that is generating lots of full updates.”

As I reported last time, object updates take three forms, Full, Terse and Delete. Full updates are generated when an object is seen for the first time, or which is undergoing a constant change in appearance / position. Terse updates are sent only when an object changes its appearance / position. Under the current Interest list system, both types of update are sent to the viewer, regardless as to where the viewer actually needs to process them. A primary part of Andrew’s work has been reworking the code so that updates – particularly terse updates – are only sent to the viewer when the object generating them is in the camera’s field-of view. This new optimisation work should further help reduce the number of updates being sent to the viewer regardless as to whether or not it has to process them.

Continue reading “SL project news: week 51” →