SL projects updates week 13/3: TPV developer meeting March 28th

A TPV developer meeting took place on Friday March 28th. The core items discussed in the meeting are reported below, with timestamps in the relevant paragraphs indicating the point at they are discussed in the video embedded here. My thanks as always to North for the latter.

Release Candidate Viewers Status

Stat Test RC  and Google Breakpad RC

[00:10] The Stat Test release candidate viewer (version 3.7.5.288371 at the time of writing) is not intended to ever move to a formal release status. Its function is to help with identifying how big an impact a bug within the Google Breakpad RC was having on viewers by comparing crash rates.

The fix contained in the Stat Test viewer is now also in the Google Breakpad RC, which was updated on Thursday March 27th to version 3.7.5.288464 (download and release notes). It’s thought that this RC is getting “pretty close to final form”, something that will likely be confirmed once the new update has been in the release channel sufficient long enough for decent statistics to be gathered in week 14 (week commencing Monday March 31st).

Interest List RC and Merchant Outbox RC

[01:00] As noted in part 2 of this report, the Interest List RC returned to the release channel wither version 3.7.5.288404 (download and release notes). This RC had been removed from the release channel on March 21st due to the number of RC viewers in the channel at that time.

The Mechant Outbox RC, which contains fixes for accurately detecting Merchant status and improves recovery for Merchant Outbox errors, was updated on Friday March 28th to version 3.7.5.288408 (download and release notes). This update means that as of Friday March 28th, all RC viewer in the release channel were up-to-date with the current release version of the viewer.

Voice RC and Sunshine / AIS v3 RC

[01:35] These two release candidates were withdrawn from the release channel on March 21st to reduce the number of RC viewers vying for promotion to release status. These have yet to be updated with the current release viewer code, however, it is anticipated that the Voice RC will be returning to the release channel early in week 14, and that the Sunshine AIS will hopefully reappear later in the week.

The binaries for the Vivox updates are currently available to TPVs for testing an incorporation, so the removal of the Voice RC from the release channel should not have unduly impacted any work being undertaken by TPVs in updating or offering Vivox 4.6.x to Windows and Mac users. Unfortunately, Vivox have yet to supply updates for Linux.

Project Viewers

Project Zipper

[01:52] The Project Zipper “fast installer” viewer was updated to version 3.7.5.288507 on March 28th, also bringing it up to parity with the current release viewer, although it remains at project status (download and release notes).

Oculus Rift

The Oculus Rift project viewer is currently on a closed beta, and so the code is not publicly viewable. The beta is reported to be going “really well”, although other sources indicate that there is a lot of dissatisfaction in how the UI works, up to an including demands that the entire UI needs to be re-thought.

Given any change in the viewer UI tends to attract a high degree of negativity right out of the gate (and also given the fact I’ve not actually seen the “Riftlook” UI), it’s hard to say how many of the concerns are objective as opposed to subjective in nature (e.g. the issue is genuinely one of ease-of-use as opposed to it being one that is considered “unusable” because it’s either not how it’s worked in the past, or requires a re-training of muscle memory) .  However, if the issues are objective and indicate significant issues of usability, it will be interesting to see how the Lab responds.

Upcoming Viewers

Group Bans

[02:24] The Group ban viewer is progressing, as noted in part 2 of this report, a new version is available through the Server Beta wiki page, and can be tested on the Morris region on Aditi. There is yet more work to be done on this, so it’s unlikely to move to an official project viewer or RC status in the short-term.

Snowstorm

A new Snowstorm RC viewer is in development, and Oz Linden is hoping to include a few more things in it. I believe this viewer may include the LSL syntax updates of STORM-1831, although the hope is apparently to get some more items into it alongside what is already there. As there are already so many viewers either in, or queuing-up for RC status, there is currently no rush to move this viewer forward.

Release Viewer

[04:02] Commenting on the current release viewer version: 3.7.4.288138, formerly the FmodEx Hotfix RC, Monty Linden indicated that there are almost a year of fixes the stream threading between the current viewer and previous versions. These include many fixes for issues where the viewer can drop a stream entirely. However, whether these include a fix for the issues where a stream containing externals ads (for cars, insurance, etc. – see FIRE-12798) can be completely dropped by the viewer and require a re-log in order to reconnection, is unclear.  However, Monty has offered to take a poke at things.

SLS Share / Facebook Photos Upload

[07:26] As reported on March 20th, the ability for users to upload photos from the viewer to their Facebook accounts was disabled by Facebook, apparently because of the inclusion of SLurl with the photo uploads.

The Lab has made changes in the viewer and in the intermediate server layer which should address Facebook’s concerns, however, Facebook has yet to unblock the upload capability at their end, and at the time of writing there has been no word on when this might be done.

The viewer-side updates have yet to reach the release viewer, but are currently in all of the release candidates in the viewer release channel.

The SL Share photo upload panel as it is in the current release viewer (), left, and as it appears in the RC viewers awaiting promotion, right. The SL option has been removed to meet Facebook's requirements
The SL Share photo upload panel as it is in the current release viewer (3.7.4.288138) on the  left,complete with the option to include a SLurl; and as it appears in the RC viewers on the right, with the SLurl option removed to meet Facebook’s requirements

Mac Issues

[08:30] A request has been made for the Lab to provide a list of Mac / Cocoa issues to help TPVs keep track of things. As reported last time this issue was raised, the Lab are hampered in trying to deal with Cocoa-related issues, as their Mac developers are working on a number of projects, and so specific issues are dealt with individually, rather than as a part of an overall project.

HTTP Issues

Monty Linden:
Monty Linden:

[19:53] The Firestorm team asked if the Lab had any data on how well the HTTP code changes were going and if they were seeing anything unexpected with server-side loads. The prompt from this question came by the fact that as downloads of the Firestorm 4.6.1 release have increased, so the number of user complaints which appear as though they might be related to HTTP have increased. Monty Linden replied:

Not that I can tell. Nothing is getting to me if there is any such story. I’ve heard the anecdotes, and whenever I’ve looked into them, the stories behind them are generally pretty random. It’s people associating recent changes with behaviour they’re seeing and assuming some correlation where there isn’t any.

Now, that said, there are some patterns that continue to persist that have existed for the past few years that I’ve been dealing with transport issues. The simulators always have had problems in the lib services with connection quotas being exceeded, and that’s still going on today. A busy region on a busy sim hots is still a problem, but I can’t say it’s worse than before. It should be better, but the data I’m getting – which is almost none – doesn’t point to any problem. I’m sorry that’s not definitive, but it’s all I’ve got at this point.

It has been reported that HTTP downloads on sims which have a lot of textures and objects, but few avatars are significantly better than HTTP downloads where there is a large number (20+) avatars. In the latter circumstance, it has also been reported that switching back to UDP appears to improve downloads. However, Monty warned against this, as UDP puts significantly more load on the simulator, which can have a noticeable impact for every user connected to that simulator.

In discussing how to confirm whether or not users really are seeing HTTP issues, Monty continued:

The first thing would be to look at the log file. If you’re getting permanent failures in texture HTTP, it’s going to show up in the log file with a fail on a final retry with an error code and a bunch of related information.So you can pretty much confirm pretty quickly whether or not they’re getting serious permanent failures for texture operations … and these will be distinct from 404s. 404s are nots founds, and you’ll see those every now and then simply because the texture reference is not valid. But other reasons for a permanent failure would be interesting.

When I updated the texture console, I put mesh errors in, but I didn’t do anything about HTTP errors, but there would be another useful area to put things in the future for monitoring error counts. but for now the log file will have the truth. Let me know what you find, because that will point us in one direction or the other.

On our services, I absolutely definitely see busy sim hosts having their input channels saturated. And when that happens, you are going to have retries, and I’ve seen some pretty hard cases of that. Some of the big social things that get put on Second Life [such as RFL] often get put on concentrated sim hosts, and these really get victimised by this problem. But again, permanent failures, they’ll be in the logs and you can look at that. Beyond that, I’ll se what i can find out.

 A problem with obtaining logs is that people will often encounter problems like this a few times and the revert back to an earlier release of the viewer. So, if you are using a viewer (SL or Firestorm) with the HTTP updates and encounters persistent rendering issues or failures when moving around SL, please consider taking a couple of minutes to fill-out a JIRA, attach your log file and submit it. Monty is very sensitive to HTTP issues, but in order to respond to them, he needs data, not anecdotes.

Other Items

AIS v3

[02:56] Oz Linden reported that ther server-side AIS v3 code will be enabled across all three RC channels (LeTigre BlueSteel and Magnum) in week 14. As BlueSteel and LeTigre had AIS v3 enabled with server update 14.03.07.287758 and Magnum with  update 14.02.21.286985, I can only assume that it was disabled again with the Magnum update 14.03.12.288004 in week 13.

Male Avatar Chest Rendering Broken / Mesh Clothes Breaking on Male Avatars

This issue related to BUG-5537, where there is a discrepancy on how the AV mesh shape is rendered/viewed by oneself and by others in the male AV chest portion, so that while an item of rigged mesh clothing looks fine to the user when viewing his avatar, other people see parts of the avatar’s chest still poking through the outfit. There has also been discussion of the issue on the forums.

The Lab is currently investigating the issue, but have no definitive answer on the problem as yet.

 

 

 

Raising the roof: HTTP gets a blog post

Those who read this blog know I try to report on the various LL projects which are on the go, both server-side and viewer side and – in some cases – both.

Monty Linden: sdpearheading the HTTP work
Monty Linden: spearheading the HTTP work

One of the latter is the HTTP project work, which has been in progress over the last couple of years and spearheaded by Monty Linden, who has been slowly but surely making dramatic changes to SL’s sometimes creaky communications mechanisms. This work started with texture fetching, way back in 2012, and has steadily progressed from there,  with changes being made both server-side and within the viewer.

Much of this work has gone unsung among the greater populace of SL as a whole, which is a shame, as Monty is perhaps one of the great heroes of SL and the Lab for taking-on this work and developing a project and roadmap which not only massively improves viewer / server communications and their overall robustness, but which is also having beneficial impact elsewhere (such as Monty rebuilding third-party libraries critical to the viewer and putting in place mechanisms to ensure they are properly maintained going forward) and also preparing the ground  for HTTP pipelining.

Monty's HTTP work encompasses viewer / server communications
An early phase of Monty’s HTTP work from 2013

Most recently, Monty’s work has involved overhauling the way in which mesh is handled between the viewer and the server (both uploads and – in particular – downloads), something which has been an issue since mesh was first introduced, due to the manner in which it effective “shotguns” the network, and also because – to a degree – people don’t fully understand the impact certain debug settings have on viewer / server communications.

The fruits of this labour have already been released server-side, and now the viewer changes are reaching a point where they will soon be filtering into viewers of all flavours, the code having now moved from a project viewer to a release candidate viewer.

(This viewer should also address the DNS problems many users have experienced and eliminate the need to use the Google DNS workaround for those who have been affected.)

The HTTP project has improved "under the hood" performance in SL in a number of areas, starting with texture fetching, anf through greater robustness of connections through the use of keepalives
The initial HTTP work raised the request rate ceiling within the viewer for texture and mesh data from A up to the blue line of C. Future work will hopefully raise it still further

The blog post is a careful and clear explanation of the work which has gone on to date, covering all aspects of the project, the positives and some of the negatives, while touching on some of the complexities of viewer / server communications which are outside of the Lab’s direct control, but which these changes may well still help alleviate to some degree. The piece also looks to the future and what also might be folded-in to the work, allowing for management decisions, staffing, and other priorities as well. While the look ahead is somewhat speculative at this point in time, it does point towards some intriguing options, such as updates to HTTP services such as inventory operations…

All-in-all, the post is a worthwhile read for anyone with any interest whatsoever in the work the Lab is putting into trying to improve Second Life and improve the experience for all of us who use it.

SL projects update 2014 week 1 (1): HTTP project viewer arrives

It’s the first week of the year and the Lab is already busy on the viewer front.

New Release Candidate: PackageFix Viewer

Thursday January 2nd saw a new release candidate viewer enter the release channel. Version 3.6.13.284995 is described as Second Life PackageFix Viewer. This update fixes an issue introduced with the 3.6.12 code base whereby the Windows executable name was changed from “SecondLife” to “SecondLifeViewer”.

As any executable using the old name (“SecondLife”) was not removed as part of the 3.6.12 installation process, problems could result when using old shortcuts pointing to the old executable, rather that the new version, which would then run and could then trigger a further auto-update, again without removing older executables or shortcuts.

With this fix, any executables using the old name (“SecondLife”) will be removed from the installation folder, so any shortcuts created to it will cause Windows to display an error message, and the user can then remove them or modify them to point to the correct executable.

HTTP Project Viewer

Monty Linden's HTTP project viewer arrives
Monty Linden’s HTTP project viewer arrives

Monty Linden’s long-awaited HTTP project viewer surfaced from QA on Friday January 3rd. Part of the ongoing HTTP project work, version 3.6.13.284698 contains three headline updates:

  • Experimental Fix for Long-Standing DNS Problems: the mechanism used to perform DNS look ups has been changed from an application-hosted DNS resolver to a combination of threads and the host operating system’s resolver.
  • Mesh Download with Better TCP Connection Behaviour: this release of the viewer reduces by 75% the number of connections used for downloading meshes. It also reuses these connections more effectively further reducing network demands. Those who have experienced connectivity problems from router instability or other networking problems may see a significant improvement.
  • General Reliability Improvements in Mesh Upload and Download: a great deal of work has been put into the mesh upload and download code. Better error handling, less frequent request retries, removal of many thread issues and fixes for numerous smaller bugs are a part of this release.

Among the changes, this viewer sees the introduction of the new GetMesh2 capability, together with its associated debug, mesh2MaxConcurrentRequests, which are designed to reduce the number of concurrent number of mesh connections from 32 to 8, but which adds keepalive functionality and improved retry logic, and which see the number of viewer / server connections which can be requested by the viewer clamped to prevent individual users overloading the server-side capabilities, as explained back in my 2013 week 36 and week 34 reports.The new capability will run alongside the existing capability as things are transitioned-in to SL.

More news on this project viewer to follow.

New Merchant Outbox Project Viewer

Arriving on Thursday, January 2nd, the Merchant Outbox project viewer, version 3.6.13.284731, is described as including fixes for accurately detecting Merchant status and improves recovery for Merchant Outbox errors. As such, it addresses the following problems:

  • Merchant Outbox viewer crashes on Exit on Windows 7 (ACME-1219)
  • User needs to click OK twice to dismiss Merchant Outbox notification (ACME-1220)
  • Merchant Outbox Initialization fails with 404 error with new account in Merchant Outbox viewer (ACME-1221)
  • Merchant Outbox panel unusable and unrecoverable if outbox folder disappears (MAINT-2287)
  • Send to Marketplace from Merchant Outbox says it fails when it doesn’t (MAINT-2301)
  • Merchant outbox with genuine failure “409 conflict” causes all further imports to fail (MAINT-2452)
  • Merchant outbox: Second Life 3.6.8 (282375) build – results when user has a merchant outbox misplaced in inventory (MAINT-3319)
  • Viewer crash when deleting trash which contains Merchant Outbox (MAINT-3320).

Those who have experienced any of the above issues when using the Merchant Outbox might want to try-out this project viewer – but do keep in mind it is a project viewer.

Related Links

SL project updates: week 49 (3): Fitted Mesh, AIS v3, Oculus Rift and more

The following notes are taken from the TPV Developer meeting held on Friday December 6th. A video, courtesy of Northspring, can be found at the end of this report. The numbers in braces after each heading (where given) denote the time stamp at which the topic can be listened-to in the video.

TPV Developer meeting (stock)
TPV Developer meeting (stock)

Release Channel Viewers

Name Updater Release Candidate

[00:17-01:40]

The Name Updater RC viewer, also released on December 3rd, has been updated to version 3.6.12.284506. This contains no functional changes to the viewer itself but contains two sets updates, hence the odd name.

The first of these is a fix for the viewer updater where problems can occur if a new update to the viewer is downloaded by the updater but deleted somehow prior to  the installer itself being executed. The second set of updates cover:

  • Changes to how the viewer packaging is done and cleans-up how the viewer channel (used to recognise the viewer and allow it to connect to the SL servers when logging-in) is distributed and established
  • Makes some changes to the viewer start-up parameters
  • Changes the package names to a uniform format which is the same for all of the operating system platforms.

The aim of these changes is to further improve the viewer build process and reduce the number of places changes have to be made in order to change the viewer channel name when building different flavours of the viewer (LL’s own or a TPV).

The RC has been performing well in terms of low crash rates, etc., and looks set to be promoted to the de facto release viewer in week 50 (week commencing Monday 9th December), and so will see-out 2013 as such if this is in fact the case.

Google Breakpad

It is possible a further Google Breakpad RC may appear in week 50.

Maintenance Release Candidate

[02:00-02:16]

The Maintenance RC viewer 3.6.12.284430, released on December 3rd  suffered an abnormally high crash rate, prompting it to be withdrawn in order for it to be looked at and crash issues diagnosed / fixed. Once these issues have been dealt with, the viewer will be returned to the release pipe.

Project Interesting Viewer

[39:24-41:07]

The Project Interesting (aka “viewer-interesting”) RC viewer has been in RC for a while and is suffering a high number of crashes, which are currently being investigated by the Lab. Unlike the Maintenance RC viewer, it has been left as an RC simply because issues are being found with it, because of both the number of people using it and the broad range of systems on which it is being run and which the Lab couldn’t possibly account for in their own testing.

At the moment, the Lab are trying to put together an update for the viewer, but they still have a couple of “pretty serious” crash issues which have yet to be resolved. However, the hope is that this may actually make it out into the world before the no change / code freeze window comes into force  on Monday December 16th, which affects all server releases and all viewer release channel releases. This would allow the updates made to get further “in the field” testing during the code freeze / holiday period.

That both the Project Interesting and Maintenance RCs are experiencing issues is something of a validation of the new viewer release process introduced by the Lab earlier this year, in that the problems being encountered with both of these viewers are not blocking the viewer pipe, unlike the situation of just over a year ago, where a series of crash issues with the old beta viewer completely halted all significant viewer updates.

Fitted Mesh Project Viewer

[02:20-03:16 / 32:05-39:20]

As noted in part 2 of this week’s report, the Fitted Mesh project viewer received a set of updates (including new avatar skeleton files) in the form of release 3.6.12.284458. The project viewer has so far received a very low number of downloads – somewhat unsurprisingly – with the total number of people using the viewer thought to be under 2,000. This means that it hasn’t as yet been used widely enough to generate meaningful crash statistics.

The response to the skeleton changes within the viewer has been “good”, and the viewer has seen a reasonable number of JIRA issues raised under the FITMESH project, etc., although the Lab cautions against anyone using the changes contained in the viewer in anything other than an experimental version of their own viewer until such time as the code reaches a Release Candidate status. The latter will not happen before the end of 2013, although there may be a further project viewer update for Fitted Mesh before the end of the year.

One thing which may happen when the viewer is approaching a release status is that it will bring with it a “significant bump” to the viewer version number, not the least of which is because users on viewers without the code may see some bizarre, or at least oddly fitting clothing on avatars using garments weighted to use the new system, as noted in my launch preview of the Fitted Mesh project.

Overall, it appears that the Lab is “pretty happy” with the way the work is developing, although they would like to see more people involved in using / testing the viewer, particularly anyone proficient in rigging mesh garments, etc, especially given the nature and state of the project, as Oz Linden pointed-out:

This is one of those times when things are in flux and can be changed… We have never made changes to the avatar skeleton casually, and we’re making a round of changes now; we’re wildly unlikely to make another round of changes for years. So if there is feedback to be had, this is the time to have it.

So if you are a creator and do have an opinion on how things might be better handled within the Fitted Mesh solution, now is the time to be involved and potentially influencing the Lab’s thinking. not every idea put forward may be taken-up; but on the other hand, waiting until the changes have been made and the viewer released will certainly mean that any ideas someone may have will have passed their sell-by date.

The Delay in Opting for this Solution

Part of the general feedback voiced when the Lab announced the Fitted Mesh viewer came in the form of questioning why it took the Lab so long to reach the decision to go with the approach. Part of the reason appears to be that mesh deformation and Server-side Appearance projects required the same expertise with the Lab to be applied to them, and so were vying with one another for manpower – and the decision was made to give the SSA project priority.

Oculus Rift Update

[24:46-26:26]

During the Server Beta meeting on Thursday December 5th, VoidPointer Linden indicated the work on making the viewer operate with the Oculus Rift headset was now “feature complete”, and that a (presumably project) viewer will be appearing “soon” with support for the headset. How soon is open to question, given VoidPointer had to be somewhat circumspect. However, following the TPV developer meeting, it appears that “soon” might actually be a little more in the realm of “later” than may be the case.

Oculus Rift viewer: "soon" probably not as close as either
Oculus Rift viewer: “soon” probably not as close as either “real soon (TM)” or “pretty soon (TM)”!

Continue reading “SL project updates: week 49 (3): Fitted Mesh, AIS v3, Oculus Rift and more”

SL projects update week 47 (3): viewer, Sunshine / AIS v3, HTTP and more

The following notes are taken from the TPV Developer meeting held on Friday November 22nd. A video, courtesy of Northspring, can be found at the end of this report. The numbers in braces after each heading (where given) denote the time stamp at which the topic can be listened-to in the video.

TPV Developer meeting (stock)
TPV Developer meeting (stock)

No Change Windows

[00:31-02:26]

Thanksgiving

As has been previously noted in this blog, week 48 (commencing Monday November 25th) is code freeze / no change window due to it being Thanksgiving in the United States. This means there was be not server deployments during the week and there will be no viewer release channel updates for the week.

Christmas

The Christmas code freeze / no change  window is scheduled to run from Monday December 16th through until Wednesday January 1st. This most likely means that once the code freeze comes into effect,  there will be no major server updates until the 2nd week of 2014 (commencing Monday January 6th, 2014), and viewer updates may be likewise.

Both the Thanksgiving and Christmas no change windows effectively mean there are only two full weeks left in 2013 in which server deployments and major viewer updates are liable to be made. However, it is possible both periods could see project viewer updates appearing. This is because any project viewer which may be available will have limited use (only those particularly interested in using / testing it are liable to run it), and so minor updates, etc., to such viewers are not seen as being potentially problematic in terms of support issues.

Viewer Updates

[00:18-00:31]

As note in part 2 of this week’s report, the release viewer was updated using the GPU table updates release candidate, leaving just the Project Interesting RC in that channel.

There are upcoming RCs in the pipe awaiting release, including an updated version of the Google Breakpad viewer and another maintenance viewer RC, while the Project Interesting viewer is to update an update as well. However, as week 48 (commencing Monday 25th November) is a code freeze week for Thanksgiving, it is unlikely there will be any releases in the viewer release channel during the week.

Fitted Mesh Viewer

[03:36-08:44]

A number of JIRA have been filed in relation to the Fitted Mesh project viewer, and are receiving attention within Linden Lab. “We’re getting the repairs together,” Oz Linden reported to those attending the TPV Developer meeting, “And when we’ve got enough of them together to do a release with, that have been tested, then we’ll do an update to that one.”

Avatar Skeleton Files

The Fitted Mesh viewer actually contains a small number of actual code changes; the majority of the changes lay within the avatar skeleton and its associated filed (e.g.  Avatar_skeleton.XML / avatar_lad.XML). This has led to speculation that other viewers can update relatively simply by using the revised avatar skeleton files. Responding to this, Oz said:

Ideally that’s true, but it turns out not to be quite completely true. It turned out that there were some code bugs that the new skeleton and weighting exposed. So there are actually some changes that will be beyond that. That is some code [to be changed].

“Adding bones exposed some limitations,” Nyx added.

One of the code fixes which is in progress appears to deal with the issue of how garments weighted to use the new skeleton appear in viewers which do not have the updates, as demonstrated in my preview article on the Fitted Mesh viewer, and shown below.

Time Frame for Formal Release

While the Fitted Mesh project viewer may well see one or more updates before the end of the year, there are no plans to progress it to a release candidate status before the start of the New Year (again, the no change windows would preclude that, at least in part).

Even with the changes now being made, the number of code changes within the viewer is “very, very small”, so when the code is in a position where the Lab is comfortable with TPVs taking it and merging it into their repositories, it should not create major issues. One thing that is not clear at this time is whether merging and incorporating the Fitted Mesh changes will be dependent upon merging other code releases coming out of the Lab, such as the Sunshine / AIS v3 code and the Project Interesting code.

Project Sunshine / AIS v23 Updates

[09:44-14:41]

Nyx Linden
Nyx Linden

Nyx Linden reports that the Sunshine / AIS v3 updates are going “really well”, and the Lab is focused on cleaning up the last few bugs of which they are aware, and it is hoped that the code will be ready for QA and then a project viewer soon, possibly prior to the December no change window coming into force.  If the viewer does make it to a project release prior to that happening, Nyx will likely hold it over until early January.

In the meantime, the Lab is still keen to get started on more extensive load testing for the new inventory service, AIS v3, using the Sunshinetest regions (1-4) on Aditi. They’d preferably like the assistance of TPVs with this, the latter having been given access to the code a couple of weeks ago so that they could start work merging it into test versions of their viewers for this purpose.

Firestorm released a version of their viewer with the new Sunshine / AIS v3 code updates to their Beta testers in week 47, although this has yet to have the legacy baking code added back into it for the OpenSim version of Firestorm.  The team is approaching this cautiously, as there is a need to try to isolate the code used in the legacy avatar baking process (which is still used on OpenSim) so that it does not interfere with / get altered by future merges with code from Linden Lab. Once this has been done, Firestorm plan to make the code available to other TPVs so that they do not have the same headache  when faced with trying to reintegrate the legacy baking code into their viewers.

Both Firestorm and Kokua (the latter having integrated the Sunshine / AIS v3 changes into a test viewer at the start of November) have indicated they are now in a position to assist with any load tests. The hope is that this will take place during December.

One of the reasons the Lab is keen to get the load testing underway is so that any remaining issues with the server-side code can be identified, investigated and fixed prior to the code being deployed on any RCs on the main grid. Any initial deployment of the server code on the main grid would likely be handled “pretty quietly”, simply because it wouldn’t be exposed to any viewers that did not have the necessary updates.

HTTP Updates

[14:41-22:12]

Monty Linden
Monty Linden

“It is currently still in QA,” Monty Linden reported in reference to his current work with HTTP 1.1 changes within the viewer. while no bugs have so far been found with the new code itself, he did reveal that the work has uncovered “quite a few bugs in mesh in general”, which are being filed internally. Currently, it is predicated that the QA round is unlikely to finish before Tuesday December 3rd, so and project viewer will not be appearing until after that date.

In addition to QA testing not finding any bugs within Monty’s code, all the numbers coming out of the performance aspects of the testing are described as “equal or better than past history”.

In the interim, Monty is continuing to lay the foundations for HTTP pipelining. As indicated in my last update on his work,  he’s been going through the third-party libraries and their repositories which are used in the viewer builds and updating them. This has led him into a number of “interesting” discoveries  as a result of tracking through all of the repository dependencies, etc., and identifying the various package mismatches and unnecessary libraries which are being packaged with the viewer (noticeably in the Linux version of the viewer), as well as one or two libraries which are not being packaged when they should be, as well as the use of multiple versions of the same library (e.g. 3 different version of Boost, 3 or 4 different versions of zlib, etc.).

Continue reading “SL projects update week 47 (3): viewer, Sunshine / AIS v3, HTTP and more”

SL projects update 44 (3): viewer, AIS, and HTTP

The following summary is taken from the TPV Developer meeting held on Friday November 1st. A video, courtesy of North, can be found at the end of this report. The numbers in braces after each heading (where given) denote the time stamp at which the topic can be listened-to in the video.

General Viewer News

As noted in part 2 of this week’s report, there are currently two release candidates in the LL viewer release channel, the GPU Table RC, which contains updates to the viewer’s GPU table but no functional changes, and another Maintenance RC, which includes finer access control for estate/parcel owners; CHUI: toggle expanding Conversations by clicking on icon  + more.

It is expected the Google Breakpad RC will be returning to the RC channel in week 45 (see below).

Several of the remaining anticipated viewer RCs / project viewers, again as previously reported, held-up as a result of issues uncovered in QA and / or bugs being re-introduced into them. These include:

  • The Group Ban List viewer: work here, which involves server and viewer changes, is held-up as a result of QA testing revealing some issues which Baker Linden is addressing (as per part 2 of this report)
  • The interest list viewer, which recently saw the issue of objects failing to render without a relog return to the code after having been fixed, and which still has one or two other issues to be fixed, although Oz Linden feels those working on it are homing in on solutions
  • The HTTP viewer updates, which were for a time awaiting QA resources (see below for a further update).

AIS v3

[02:29-07:07]

Nyx Linden, Tiny RobotTM, is in the Halloween spirit
Nyx Linden, Tiny RobotTM, in the Halloween spirit

The Lab is keen to start progressing this work towards a release. As with Server-side Appearance, they’re looking to TPVs to help with various aspects of testing.  To this end, a request has been passed to TPVs that they indicate to the Lab when they have merged the code into experimental versions of their viewers so that a pile-on test can be arranged in order to put the updates through their paces.

There is no specific date for when this will take place, and commenting on the project in general, Nyx Linden said:

Now is a good time to start your merges, I’ve just pushed an updated to Sunshine external, so you guys should have our latest and greatest … But again, this is not formally QA’d, we’ve been testing things as we’ve been going on, but it is not ready for release yet. But now is a good time to start doing test merges and getting side branches up-to-date with that.

The latest code includes a fix to viewer-side behaviour. On logging-in to Second Life, the server sends a list of the things it believed an avatar was wearing, although the message only had room for one wearable of each type (e.g. undershirt, shirt, jacket, etc.), and so it may or may not be up-to-date with the Current Outfit folder.

While the current release versions of the viewer ignore the contents of the message, they do still wait on the message for timing (thus slowing down avatar processing). With the new code, the timing pause is being done away with, so that the viewer should be able to start resolving the avatar from the Current Outfit Folder whether or not the message has been received. There is a slight side-issue with this change that may affect some avatars under limited circumstances, but a fix for this issue is due to be made available to TPVs before the code even reaches any experimental versions of their viewers.

Viewer Crash Reporting

[09:00-14:50 and 26:03-31:15]

There is an issue with the viewer crash reporting which means that a lot of crashes are being incorrectly reported as viewer “freezes”. This is something the Lab is aware of and is working to address. The problem lies with a number of the mechanisms used to determine various types of crashes are not working, with the result that the associated crashes are being misreported as the viewer freezing.

As well as addressing this issue, the Lab has also been working in other areas related to Google Breakpad and crash reporting, including:

  • Simplifying and cleaning-up the creation and interpretation of the marker files used to generate crash rate numbers
  • Re locating these files much earlier in the viewer initialisation and log-out processes so that crashes which occur during the viewer’s initialisation or termination can also be captured
  • Addressing those crash reports which are generated, but lack associated stack dumps or mini-dumps and ensure that in the future that do have the required information, thus allowing the Lab to fill-in more of the blanks and ensure even more meaningful data is gathered as a result of crashes.
TPV Developer meeting, Friday November 1st
TPV Developer meeting, Friday November 1st

It will be a while before this work is ready for inclusion in viewers; one reason for this is because the improvements to Google Breakpad require continual rounds of user testing as changes are made (hence why the Google Breakpad RC appearing and vanishing and reappearing in the viewer release channel). However, once the code is ready for release, it should provide for more accurate crash reporting across all viewers. As the work comes to fruition, it should allow for more accurate identification of a range of crash situation and assist with the work in trying to eliminate them.

Continue reading “SL projects update 44 (3): viewer, AIS, and HTTP”