Tag: Server Updates

SL project updates: week 10 (2): server, SSB, materials and SSAO

Posted on by Inara Pey

Server Deployments Update

All of the deployments planned for week 10 went ahead as scheduled. While further issues related to region crossings have been reported, these are not thought to be related to any of the new code deployments for this week (see below for more).

The one issue that has been noted with the deployments is for VWR-786, which formed a part of the Magnum deployment.

This was supposed to ensure that if a friend does not have ‘See my online status’ permission, they will now see “User is not online ..” message following IM or inventory offer. However, the result has also been that if you IM a non-friend, the server always returns the “User is not online” message. The short-term solution for this is to remove the change from week 11’s releases in the interest of getting the other fixes (BUG-1612 and SVC-8019) across the grid.

The Lab is particularly keen to see SVC-8019 deployed to the entire grid, as this should fix issues of regions not handshaking correctly with one another following a rolling restart. The cause of this is believed to be due to regions looking at stale cached copies of a neighbouring regions’ status. With the update, regions grab more up-to-date copies of the status of their neighbours.

Server-side Baking: Further Pile-on / Load Test

Nyx Linden has announced that there will be a further SSB pile-on / load test on Thursday March 14th, following-on, as with the last test, from the Server Beta meeting on Aditi. The test is liable to be in much the same format as the first test, of which Nyx notes, “It gave us a lot of information and we’ve been working on a number of fixes, both to Aditi inventory, as well as viewer and back-end changes. Given this, the reason for the next test, in Nyx’s words, is because, “We’d like to see how much progress we’re making.”

The first SSB pile-on / load test (image courtesy of Latif Khalifa
The first SSB pile-on / load test (image courtesy of Latif Khalifa

Those wishing to participate will be required to be using the latest version of the Sunshine project viewer (3.4.5.271419), and are advised to attend the Server Beta meeting on Aditi ahead of time (the meeting commences at 15:00 SLT on Thursdays). Addressing those who participated in the first test, Nyx added, “If you had trouble at the last pile-on with outfit switching, feel free to test out the new build in advance – you should be able to comment on the JIRA tasks you filed, or email me directly with any issues.”

Materials Processing

Following-on from his replies to my question at the open-source dev meeting on Monday March 4th, Oz Linden talked some more on the status of the materials processing project at the Wednesday meeting on March 6th, “There’s one build floater bug and one crash that need fixing,” he said in kicking-off the discussion on materials, “I might even be willing to let it out without the crash fix (though it’s pretty bad).”

However, before everyone starts shouting, “Yes, yes!” :), even the release of a crashy version of the project viewer requires the “other” problem to be fixed. This appears to be related to a texture list getting corrupted, and which can manifest in a number of ways, including:

  • The normal map picker reverts to displaying the diffuse (texture) map after a normal map has been selected and the picker closed, with the normal map failing to render on the object / face it has been applied to
  • If deferred rendering is turned off, anything using materials appears black
  • If bump mapping is disabled, objects using materials appear to randomly adopt nearby textures (including skin textures) which can change as the camera is rotated / moved.
Normal map application issue: a normal map is selected and apllied to an object face (l); however, on re-opening the build floater, the map appears to have reverted to the diffuse map (r), and the object face does not render as expected
Normal map application issue: a normal map is selected and applied to an object face (l); however, on re-opening the build floater, the map appears to have reverted to the diffuse map (r), and the object face does not render as expected

Continue reading “SL project updates: week 10 (2): server, SSB, materials and SSAO” →

SL projects updates: week 10 (1): server, materials and SSB

Posted on by Inara Pey

Server Deployments Week 10

A full set of server deployments this week.

On Tuesday March 5th, the Second Life Server (SLS) channel received the server maintenance project that was deployed to all three RC channels in week 9. This update only contains a fix to a single crash mode.

On Wednesday March 6th, the three Release Candidate channel should receive the following code deployments:

  • BlueSteel and LeTigre: a new server maintenance project, which fixes a fairly common crash mode, together with Baker Linden’s large (as in file size) object rezzing project aimed at improving simulator performance (see below)
  • Magnum: a new server maintenance project, which includes a mix of bug fixes and stability improvements. Specific fixes mentioned in the release notes are:
    • BUG-1612: region Owners and estate managers finding they are unable to teleport back to their region after disabling direct teleports to the region
    • SVC-8019: region visibility delays following region restarts. This may help with the problem of diagonally adjacent regions failing to render
    • VWR-786: if a friend does not have ‘See my online status’ permission, they will now see “User is not online ..” message following IM or inventory offer.

Large Object Rezzing Project

Baker Linden has been looking to improve how objects with large file sizes are handled by the simulator software when being rezzed. He describes the work thus, “What I’ve been working on is hopefully significantly decreasing lag spikes when rezzing large, complex objects [such as those with lots of scripts]. Large does not necessarily imply size, but size of the files being read. When an object is rezzing, we have to parse the object / mesh files and create our in-world objects with that data.”

Until now, reading and parsing of any files related to objects which require rezzing has been on the main thread. When several such objects requiring rezzing at the same time, the simulator stalls. Baker has been moving the reading / parsing operation to a background thread in the expectation that this will prevent the simulator being choked.

The key point about this work is that it is specifically aimed at preventing the simulator processes from choking and a region stalling when there are a number of large object files being read / parsed, not at actually “speeding up” the physical rezzing process. As such, it is unlikely that objects will appear any faster in people’s in-world view as a result of this work. However, what it does mean is that the simulator code will be better able to handle rezzing multiple “large file” objects without the attendant region lagging which can occur as a result of the simulator being unable to process messages from viewers and other simulators, etc.

Materials Processing

In my last update on this work, I reported that the Lab believed they had one more issue to resolve with the materials processing project, after which the way should be clear for a project viewer to be made publicly available. At the time, it wasn’t clear exactly what the problem might be. However, on Monday March 4th, I was able to ask Oz about the problem, and it appears that it is with the project viewer itself.

“We’ve got a viewer, but it’s so crashy, and the crashes are mostly in material property editing, that I don’t want to distribute it yet…. I’m concerned that doing so would result in a lot of broken content lying around,” Oz informed me.

Materials processing: viewer issues delaying project viewer release (image courtesy of Geenz Spad)
Materials processing: viewer issues delaying project viewer release (image courtesy of Geenz Spad) – click to enlarge

I asked Oz if the crash problems were related to physically applying maps to objects and / or object faces. He confirmed that this is indeed the case – and that the latest (non-public) version of the project viewer can crash if even the parameters for maps applied to an object / object face are modified. However, he went on to say, “Hopefully we’ll get the worst of the crashes dealt with soon, and then we can start giving it to a wider audience. We’ve already solved a bunch of them, but it’s not quite ready for even open alpha testing.”

So, for those who commented on the lack of any update following my last SL project update from week 9, I’m afraid the situation still appears to be one of, “Hurry up and wait.”

Continue reading “SL projects updates: week 10 (1): server, materials and SSB” →

SL project update: week 9 (2): servers, HTTP, SSB

Posted on by Inara Pey

Server Deployments – week 9

Server Life Server (Main) Channel

As noted in part 1 of this report, the Second Life Server (SLS / Main channel) received the server maintenance project which was on all three RC channels in week 8, comprising miscellaneous bug fixes, and the improved region restart notification. Deployed on Tuesday 26th February, the roll-out was followed by a number of issues being reported via the deployment forum thread.

Issues have included:

  • Parcels / regions vanishing from search for a period of time following the restart (an issue WolfBaginski Bearsfoot gave some thought to in commenting on part 1 of this report)
  • Further region crossing issues, with loss of control followed by “broken” camera positioning after recovery. This issue has been reported at some of the Simulator and Server Beta meetings previously, and Eric Gregan has produced a video demonstrating the problem as it occurs on some aircraft, although he’s also come up with a means of avoiding the issue which may or may not work for people.

  • Problems with Dwarfins attachments.

Release Candidate Channels

On Wednesday February 27th, all three Release Candidate channels received a new server maintenance package which includes a fix for a crash mode (see the release notes (BlueSteel)). No significant issues have been reported for this deployment.

SL Viewer

The Communications Hub User Interface finally made its debut in both the SL development and SL beta servers, arriving as release 3.5.1.270826 for the development viewer on Tuesday 26th February, and 3.5.0.270825 for the beta viewer on Wednesday 27th. I’ve provided a brief introduction to CHUI, and there is also a video from Torley Linden.

CHUI is liable to remain in beta for a “Good long run”, to quote Oz Linden. Hopefully, this may mean that materials processing will be the next significant viewer update to arrive as a project viewer.

Interest List

The recent updates to Andrew Linden’s interest list work are apparently won’t reach a RC channel in week 10, but should see deployment in week 11 (week commencing Monday 11th March). This work includes fixing issues with avatar appearance teleporters which use llSetRegionPos(), as well as correcting problems where objects which change appearance behind the camera ‘snapping’ into place when you rotate the camera back.

Server-side Baking

On Friday March 1st, Linden Lab unexpectedly released an updated version of the SSB project viewer (release 3.4.5.271118), which includes their approach to overcoming the problems of the Z-offset capability common to many TPVs being broken as result of the SSB code. The new approach, which I was able to outline when it “launched”, introduces an additional shape slider into the Edit Shape floater; as such, the ease-of-use of the approach, particularly for those who may swap between different avatar shapes (e.g. “normal” and petite) is very questionable.

The new "Hover" option in the Edit Shape panel for adjusting avatar height offsets in the Sunshine project viewer. Not the most elegant solution
The new “Hover” option in the Edit Shape panel for adjusting avatar height offsets in the Sunshine project viewer. Not the most elegant solution

Continue reading “SL project update: week 9 (2): servers, HTTP, SSB” →

SL project news 9 (1): servers, SSB, materials and Aditi

Posted on by Inara Pey

Deployments for Week 9

It’s a “lightweight” week for deployments, with all channels receiving one of two maintenance packages.

Server Life Server (Main) Channel

Region restart notifications: the "old" stayle (top), and the "new" style (bottom)
Region restart notifications: the “old” style (top), and the “new” style (bottom)

On Tuesday February 26th, the Second Life Server (SLS / Main channel) received the server maintenance project which was on all three RC channels last week.  This project has miscellaneous bug fixes, along with an impoved restart notification (which will only be apparent when this new version restarts for the next update) – see the image on the right.

While this is much better than the previous notification, it does suffer from the fact it will still fade from the viewer’s window and become a chiclet whether or not the OK button is clicked – and thus can still be missed if attention is not on the screen when it appears. To help avoid this, it is possible to set a sound for the alert using the Debug Setting UISndAlert, which requires the UUID of the sound, which coule be one you’ve uploaded to inventory yourself (Firestorm additionally has UISndRegionRestart, which can be set through Debug Settings or Preferences > Sound & Media > UI Sounds 2).

This release  means the until the RC channel deployments take place (see below), almost the entire grid is running the same server code – the only exceptions being a few “special” regions (such as the Linden Realms regions) – release notes.

Release Candidate Channels

On Wednesday February 27th, all three Release Candidate channels should receive a new server maintenance package which includes a fix for a crash mode – release notes (BlueSteel).

As always, there is a forum thread for discussing the deployments.

Server-side Baking

Following-on from his initial feedback given at the TPV Developer meeting on Friday 22nd February, Nyx Linden gave a further update on the recent Sever-side Baking (SSB) pile-on / load test when chairing the Content Creation User Group meeting on Monday 25th February

Thanks all for everyone who helped out with the pile-on test of server appearance last Thursday. The results tended to be that when the system worked, it was nice and speedy, however there were a number of issues that prevented it from working for everyone all the time. We got a good bit of data on the bugs we knew were there as well as a couple new ones. we’ve been crunching on the results and are starting to attack the issues, so it was definitely successful in achieving what we wanted: more data :). There will be future rounds, so thanks in advance for anyone who comes back to help us test again.

Z-offset Issue and SUN-38

As noted in the comments following my quick SSB test using currently available TPVs with SSB enabled, there is an issue with the “Z-offset” capability common to many third-party viewers, which allows the vertical height of an avatar above the ground to be adjusted, such that sits and kneels don’t leave the avatar apparently floating in the air, and which allow those with very tall / giant avatars or very small / petite avatars and those wearing full body mesh to similarly adjust their vertical placement relative to the ground / floor.

The problem was reported to Nyx in week 8, and SUN-38 was subsequently raised by Henri Beauchamp on February 24th to officially log the matter. Commenting on the problem during Content Creation meeting, Nyx Linden stated. “It hasn’t escaped our notice. We’re considering a couple different approaches … we’re considering a few different options. Suggestions appreciated, but we haven’t officially settled on an approach we’re going to commit to publicly just yet.” He also suggested that any questions on the subject be asked again next week, presumably by which time LL will have had time to consider the problem in detail, and hopefully consult with TPV developers.

Interest List Updates

Andrew Linden is continuing to work on the interest list code, with more refinements to come, as well as looking at a number of bug fixes. The updates about to go to LL’s QA include changes which should make scenery load faster  / more correctly when first logging-in to SL or teleporting to a region.

Aero Pines Park
Future updates to the interest list code should see scenery in your world view rezzing a lot faster

Continue reading “SL project news 9 (1): servers, SSB, materials and Aditi” →

SL project news 8 (2): servers and issues

Posted on by Inara Pey

Update February 24th: Metabolt 0.9.64.0 (Beta) was released on February 23rd to address the issue of nearby objects not being recorded following theserver-side interest list updates.

Update February 22nd:  Radegast 2.8 has been released, which both provides support for server-side baking and resolves the interest list releated issue of failing to correctly report nearby objects noted in the first poart of this report.

Week 8 Main (Second Life Server / SLS) Channel Deployment

The expected Main channel (SLS) deployment took place on Wednesday 20th February, as anticipated. The SLS channel deployment came a day late as a result of the Presidents Day holiday in the US – Main channel deployments will be back to their normal Tuesday slot from week 9 (week commencing Monday 25th February.

Issues

  • There is a report that both the Radegast and Metabolt text clients are either incorrectly seeing local objects, or failing to see them at all, possibly as a result of the interest list code deployment.
  • There is a bug in the  interest list code which means that child prims within a linkset do not always render from a distance – you need to cam in towards them for them to appear, and they can then vanish again on camming away. A fix for this is expected in the “next round” of interest list updates.

Week 8 Release Channel Deployments

There have been signficant changes to the RC channel deployments occurring on Thursday 21st February (again a day late due to Presidents Day).

Due to a last-minute bug being found in Baker Linden’s large object rezzing code updates, the package intended for both BlueSteel and LeTigre has been pulled. Instead, these channels will receive the code targeted for Magnum, together with the updates made to the Main channel.

Diagonal Region Rendering Issues

I’ve reported on this on a number of occasions recently, and JIRA SVC-8130 is still open on the matter. The incidence of occurrences seems to be on the increase, with more people having anecdotal evidence of problems. Following his comments in the second part of my week 7 report, Simon believes a fix is in the works, with Simon reporting at the Simulator User Group meeting on Tuesday 19th February that it is, “With QA now.”

Missing regions: bug fix on the way ...
Missing regions: bug fix on the way …

Region Crossing Issues

Numerous problems are being reported with region crossings, some of which appear to be on the increase of late.

Content “Warping”

I first reported on this in week 48 of 2012, as one of a number of region crossing issues being reported. When crossing between regions, the contents of the region you’ve just left suddenly “warp” into the region you’ve just entered, appearing for a time and perhaps somewhat deformed.

When the matter was first raised, Andrew Linden thought it might be more a server-side issue than a viewer-side problem – although all the issues raised at that time seemed to be split between possible issues within the server code and the viewer code.

warp-4
Sometimes on crossing between regions, objects from the region you’ve just left seem to appear “warp” into the region you’ve just entered, and may appeared deformed…

In looking at the problem as a server-side issue, Andrew had hoped that the matter would be resolved via the interest list code roll-out. However, it has persisted despite the interest list code having been deployed to an RC channel, and is now largely regarded as a viewer-side issue. The status of any fix for the problem is unclear.

...only to disappear after a few seconds / as you approach them
…only to disappear after a few seconds / as you approach them

Camera Position Lost on Region Crossings

There has ben a return / increase in issues of the camera position getting “messed up” when crossing between some regions in a vehicle. This can take a variety of forms, most notably with the camera suddenly shifting to show a side view of the vehicle while driving, rather than the usual forward-looking view SVC-7684 provides the details, complete with a video demonstrating the issue. It appears that fixes for this problem have thus far been on a per-region basis, as there is no clear indicator as to the underlying issue – although it is recognised enough such that Motor Loon now incorporates a scripted workaround for the problem in lieu of a fix, which forces the correct camera positioning to be re-applied on a script crossing. The Lab have been trying to investigate the issue, but are having a hard time reproducing it in a consistent manner. Those routinely experiencing the problem on given region crossings should consider filing a JIRA specifying the issue and region crossings where it occurs and reference SVC-7684.

Increase in Crossing Disconnects and Other Issues

The Simulator User Group meeting on Tuesday 19th February saw an increase in the number of other region crossing issues, with people reporting several issues, ranging from a disconnect on attempting a region crossing through to an increase in teleport failures when moving between issues and getting stuck in the corner of a region and spinning during a region crossing. Again, it is unclear what may be causing these issues, some of which have been reported as increasing somewhat sharply over the last 2-4 weeks.

Sudden Region Lag

It has been a while since I last updated on this issue, as reported in both a forum thread by Toysoldier Thor and others and in a JIRA BUG (BUG-355). In sort, the problem remains, with live event venues reporting issues, and the Lab uncertain as to the cause. When the issue was raised at the Simulator User Group meeting, Simon Linden replied, “Hmm, that sounds like it needs a fresh look. It seems like there’s a spike in networking traffic, performance goes bad and thus all sorts of things don’t work well.”

Interestingly, the four regions of the recent OBR in SL event did not suffer any such issues, despite a constant and high volume of traffic, although these had apparently received additional attention from the Lab to help ensure problems might be minimised.

Server-side Baking Load Test

A final reminder that there will be a Server-side Baking load test on Aditi on Thursday February 21st, following the Server Beta meeting. For details, please see my original announcement.

SL project news – week 8 (1): server releases, viewer, SSB load test, interest lists

Posted on by Inara Pey

Server Deployments – week 8

The deployments scheduled for the week commencing Monday 18th February are detailed below. Please note that due to Monday 18th being a holiday in the United States (Presidents Day), the deployments will be taking place one day later than usual.

Main (Second Life Server / SLS) Channel

The Main channel should receive the following two projects:

  • The Interest List Improvement project, which has been on the Magnum RC channel for the past few weeks
  • Server-side support for materials processing.

Note that there is still no publicly available project viewer to take advantage of the materials project code, although there may be news on this later in the week.

This deployment should take place on Wednesday 20th February – release notes.

Bluesteel and LeTigre Release Candidate (RC) Channels

Baker Linden in a change from his usual rooster avatar
Baker Linden in a change from his usual rooster avatar

Bluesteel should receive Baker Linden’s object rezzing code changes, which were reported here in week 1. These updates have nothing to do with the interest list code changes. Baker describes the aim of the work as, “Hopefully significantly decreasing lag spikes when rezzing large, complex objects. Large does not necessarily imply size, but size of the files being read. When an object is rezzing, we have to parse the object / mesh files and create our in-world objects with that data.”

Until now, reading and parsing of any files related to objects which require rezzing has been on the main thread. When several such objects requiring rezzing at the same time, the simulator stalls. Baker has been moving the reading / parsing operation to a background thread in the expectation that rezzing multiple “large” (again, in terms of file size, not the size of the object itself) objects will not choke the simulator, causing performance issues.

These deployments should take place on Thursday 21st February – release notes (Bluesteel).

Magnum Release Candidate (RC) Channel

Magnum should receive the same maintenance server update deployed in week 7 to LeTigre, intended to fix miscellaneous crash modes.This deployment also includes the following:

  • An improvement to the rolling restart notifications so that they appear in an alert format (as with manual region restarts) rather than an easily missed notification. This change will only be apparent in restarts following the code deployment restart (as per JIRA SVC-7759)
  • A fix to an encroachment / return problem:  if you’re banned from the neighbour’s parcel, you couldn’t select  / return items that encroached on your parcel (see JIRA SVC-496)
  • Instant messages are now truncated to 1024 bytes to prevent certain types of delivery failure. Currently, the IM database supports larger messages than the delivery system can handle. This change will enforce a limit of 1024 bytes when processing messages coming into the database as well as those being sent out.

This deployment should take place on Thursday 21st February – release notes.

SL Viewer News

The release version of the SL viewer moved to the 3.4.5 code base on the 14th February, with the release of viewer 3.4.5.270263 (release notes). At the same time, the Server-side Baking project viewer received its second update with the release of version 3.4.5.270409, od which more below.

The CHUI viewer received a further update to the development version, reaching 3.4.6.270520 on February 18th. This project is currently the next in line to merged into the viewer-dev code base (development viewer) and then into the beta code base.

Server-side Baking Load Test

Serer-side baking - load test February
Serer-side baking – load test February 21st

A reminder that if all goes according to plan, there should be a special load test for Server-side Baking on Thursday February 21st, and volunteers are being sought to assist.

This will take place on the SSB test regions on the beta grid (Aditi), immediately following the Server Beta User Group meeting which take place at 15:00 SLT on Thursdays in  Morris, also on Aditi with the aim of placing the SSB code under a stress test representative of the loads it will face when deployed to the main grid, with people routinely changing outfits, updating their appearance, enter / leave regions running the SSB code (given that the grid will, for a time, be running both the current avatar baking service and SSB as the latter is initially deployed), and so on.

Test Requirements

While final details of the test have yet to be confirmed, key requirements for those wishing to participate in the test are as follows:

  • Participants must be able to log-in to Aditi and attend the Sunshine test regions from 16:00 SLT onwards (participants can attend the Server Beta UG meeting ahead of the test if they wish)
  • Participants must be running the latest version Server-side Baking project viewer (version 3.4.5.270409 or later) – this viewer has been specifically configured to report statistics required by LL for the test
  • Participants should have a number of outfits of system clothing, preferably with multiple layers, which they can swap between during the course of the test. Library outfits are acceptable, but LL are keen for people to use their own outfits to add greater weight to the tests
  • Clearing the viewer cache prior to the test is suggested, but not an absolute requirement.

“if you have specific failures we’ll ask for your viewer logs, otherwise just running through the test will help us gather data,” Nyx added when explaining what is required by way of feedback from those opting to take part.

Further details can be found in my SSB Load Test announcement.

Continue reading “SL project news – week 8 (1): server releases, viewer, SSB load test, interest lists” →