Firestorm: where next and early looks

Update February 25th: As per a comment from Jessica Lyon, Firestorm have now merged the Server-side Baking code and updates to RLVa into one of their private repositories.

Update, 19th February: A transcript of the core part of the meeting, including Jessica’s Firestorm status overview and the Q&A session, complete with the video recording, is now available.

Update, 14th February: The initial video recording of the meeting is now available on YouTube, and an HD version will be available soon.

firestorm-logo

Wednesday February 13th saw the Firestorm team host an open meeting to discuss what is happening vis-à-vis Firestorm given all the various ongoing viewer-related projects currently underway (CHUI, materials processing, server-side baking, plus Firestorm’s own updates and improvements) – and when the next release is liable to hit the public at large,

Several members of both the Firestorm development and support teams were on-hand to field questions, with Project Lead Jessica Lyon leading things off with a 15-20 minute overview as to what is happening, where the viewer stands at this point in time, what the plans are for the immediate future and what we might expect to see in Firestorm in upcoming releases.

The Short Version

  • The Good:
    • Firestorm will be supporting all of the new viewer capabilities coming out of LL, although CHUI will require careful consideration as to what is adopted and how, as Firestorm already offers several similar options to those being added to the viewer by CHUI
    • Firestorm will be getting a range of new features (although not all at once) which include: further work on re-implementing legacy search capabilities, the ability to save and reload personal settings; more OpenSim support; new windlight settings; new UI skinning; further work on adding v1-style functionality
  • The Not-so-good:
    • Serious crash and other issues have also come to light in merging Firestorm with the latest LL 3.4.5 code which the team are endeavouring to resolve
    • Server-side baking (SSB) is the priority for the Firestorm team at present (as it is with other TPVs), as it has a major impact on how people will see things in-world, and it is the project which LL are emphasising. However, integration of the SSB code into TPVs (particularly those supporting RLVa) is not proving easy
    • The emphasis on work at the moment is overcoming bugs, issues and problems and trying to get Firestorm to a point where it is running the SSB code.

Taken together, the latter points mean that while a new version of Firestorm is in development, there will be something a further wait before it appears, and when it does, it my not have such a huge range of new features as has been found in previous releases and might suffer from stability issues.

Jessica Lyon
Jessica Lyon (seated centre, at the edge of the stage) with members of the Firestorm development and support teams, discusses Firestorm on Wednesday February 13th

Viewer Status

There are some serious issues within the Firestorm development code which are delaying progress towards a potential release. Firestorm has been merged-up to the Linden Lab 3.4.5 viewer code, and this has given rise to some severe problems for Firestorm (and is actually having an impact on other projects, as I reported earlier this week).

Commenting on the situation, Jessica Lyon pulled no punches, stating:

I’m going to be completely honest with you guys. Right now Firestorm, for us internally, is in pretty bad shape since our merge with Linden Lab’s TIP (3.4.5 code). There are a lot of bugs that we’ve inherited; there’s a lot of regressions which we’ve inherited. Ed [Merryman, lead for Firestorm Support] is crashing about two times a day – and for those of you know Ed, know that Ed never crashes. So if Ed is crashing on our recent builds, we’ve got some problems. We’ve got some log-out crashes, log-out things; log-in crashes … Basically, we’re not in great shape, and we’ve got a lot of fixing-up to do before we’re ready for a release.

As well as inheriting bugs, the merge has also highlighted bugs and issues within the Firestorm code itself which also need to be fixed. All of this adds up to recent builds for the viewer being “way worse” than the current release version in terms of stability and issues, and it is going to be a while before these issues are fully resolved.

Server-side Baking

Server-side baking is perhaps the most prominent viewer project underway at the moment, inasmuch as it is essential that all viewers connecting to Second Life be able to support it in order to avoid in-world experiences from being broken. Simply put, avatar skins and system clothing will not render on viewers which do not support SSB once the code is fully deployed, as shown below. )Things are somewhat more involved than that, and for those unfamiliar with the project, I’ve covered it in-depth in Avatar Baking: “and the clock has started!”. )

The SSB problem in part: I'm stabding on an SSB-enabled region. On the left  - as I appear to others who are using an SSB-enabled viewer; On the right, as I appear to others who are using a viewer which does not support SSB.
The SSB problem in part: I’m standing on an SSB-enabled region. On the left – as I appear to others who are using an SSB-enabled viewer; On the right, as I appear to others who are using a viewer which does not support SSB.

As it stands, Firestorm has yet to be merged with the Lab’s supplied server-side baking code for the viewer, although work has been underway within the team in a separate repository to the 3.4.5 code merge. A major problem here, as I again reported earlier this week, is that SSB has considerable (and negative) impact with RLVa. These problems are compounded by the fact that the test regions for SSB functionality are all on Aditi, which has considerable issues of its own at the moment, which are affecting people’s ability to reliably test code, and all have scripts disabled – which makes testing RLVa fixes alongside SSB somewhat difficult.

Currently, the Lab remains sympathetic to the issues TPVs are facing (and have offered help wherever practicable), and are not currently pushing a date by which TPVs must be ready for SSB to go live. They’ve also acknowledged that some of the problems TPVs are facing are down to delays on the Lab’s part, such as not making any bug fixes to the viewer code available until January 30th, some seven weeks into the planned eight-week window in which it had been hoped TPVs would be able to integrate the code. However, it is clear that TPVs are feeling under pressure to get SSB-capable versions of their viewers sooner rather than later.

Please use the page numbers below to continue reading this article

SL project news week 7 (1): Server deploys, viewer updates

Server Deployments for Week 7

On Tuesday February 12th, the Main channel (Second Life Server / SLS) received the maint-server package focused on crashes fixes which was  deployed in week 6 to LeTigre – release notes.

On Wednesday February 13th the three RC channels are scheduled to receive the following packages:

  • Bluesteel: retains the materials processing project code  – please see the SL Viewer section below for information regarding the current status of viewer development work for thi project – and should receive the same fixes and updates being deployed to the SLS channel (above) – release notes
  • Magnum: retains the interest list project code and should receive the same fixes and updates being deployed to the SLS channel (above) – release notes
  • LeTigre should receive a new maintenance server update to fix miscellaneous crash modes – release notes. This deployment also includes the following:
    • An improvement to the rolling restart notifications so that they appear in an alert format (as with manual region restarts) rather than an easily missed notification. This change will only be apparent in restarts following the code deployment restart (as per JIRA SVC-7759)
    • An avatar banned from a parcel can now select an object located in that parcel if they have ‘Return’ powers over that object (see JIRA SVC-496)
    • Instant messages are now truncated to 1024 bytes to prevent certain types of delivery failure. Currently, the IM database supports larger messages than the delivery system can handle. This change will enforce a limit of 1024 bytes when processing messages coming into the database as well as those being sent out.

As always, a forum thread has been created for discussions / feedback on the deployments.

SL viewer Updates

Beta Viewer

While the current beta release of the 3.4.5 code remains stable, problems are being reported in merging various project code updates with the 3.4.5 code base, some TPV developers are reporting issues attempting to merge the 3.4.5 code into their viewer builds, notably with Linux 64-bit and Windows, although the same issue does not appear to be occurring with Mac OSx.

In the meantime, one more 3.4.5 beta release of the SL viewer is anticipated during week 7.

CHUI

As noted in part 3 of the week 6 report, the CHUI project viewer is now largely on a par with the development viewer, although it has yet to be actually merged with the main viewer-dev code trunk. This is due to happen very soon. In week 6, Oz Linden commented that it is anticipated that CHUI will be the first of the upcoming projects to merge with the beta viewer code, and would then be in for a (quot) “Good long run” in beta. This week, it has been hinted that CHUI could actually have a lengthy run in both the development viewer and the beta viewer as final priorities are sorted out.

Server-side Baking Project Viewer

Problems remain in merging the SSB code with viewers using RLVa, although good progress is being made in addressing them, although they have yet to be merged-up to a TPV which supports RLVa and have a version under development using the SSB code.

In the meantime, the latest updates to the viewer code from the Lab were pushed to the project repositories on Monday 11th February, and a new version of the Sunshine project viewer for Windows emerged on Tuesday February 12th – Sunshine 3.4.5.270231, with Mac and Linux builds to follow.

Materials Viewer

Problems remain with the materials processing viewer-side code. Precise details are not clear, but the latest private (LL and project members only) Windows builds are being reported as being, “Conspicuously more unstable than the Mac builds”, in that they tend to crash when connecting to a materials-enabled server. Investigations into the problems (notably by TPV developers engaged in the project) are ongoing to try to locate the underpinning cause.

Materials processing: viewer issues still being resolved, so no public project viewer as yet
Materials processing: viewer issues still being resolved, so no public project viewer as yet

There are a couple more features to go into the materials viewer as well prior to it reaching a public project viewer status, but these are apparently being held-off until the current server interaction problems are resolved.

Experience Permissions

July saw the launch of the first phase of the Advanced Creation Tools, also referred to as experience tools. Following problems with an initial deployment of the tools in June, which resulted them being exploited as a means of griefing, the “first phase” of the release saw the tools implemented with existing permissions system in place, with the intention of updating the permissions system to allow the tools to be more fully used “in the future”. Since then, it has been something of a waiting game to see if / when the promised new experience permissions would be deployed, and my own last report on the general status of the permissions system being in December 2012.

Questions are still being asked at various LL meetings, he most recent being tabled at the Content Creation User Group on Monday 11th February. In reply, Nyx Linden indicated he would attempt to get an update from Falcon Linden, who is leading this project.

Object Inventory Issues

Some people may have encountered problems when trying to transfer a number of inventory items into a box (prim rezzed in-world) or similar. There are two known issues with this:

  • The first is (non-publicly viewable) JIRA BUG-383: “When HTTP Inventory is disabled, mass selecting a bunch of inventory items and attempting to add to an objects contents will fail”. This will apparently happen if the HTTP inventory setting in a viewer is changed without cache also being cleared
  • The second is (non-publicly viewable) JIRA SEC-1041, which is apparently (according to Kelly Linden) related to “An internal TCP error between sim and dataserver.” The problem manifests with the in-world error-message “Inventory creation on in-world object failed” on trying to transfer a large number of items (e.g. 50+) from inventory into the contents of a prim in one go.

As the JIRA on both of these issues are non-viewable, their status isn’t clear, other than the problems described in them can be reproduced. SEC-1041 is the more serious of the two problems and might result in a crash, although it will not result in physical damage to a person’s inventory.

Firestorm Q and A: 13:00 on the 13th

firestorm-logoJessica Lyon, project lead for the Firestorm team, has announced that there will be a limited capacity meeting on Wednesday 13th February at 13:00 SLT, to discuss the Firestorm project. The core part of the announcement reads:

Every once in a while, and probably not often enough, we host a Question and Answer session in Second Life for the Second Life community. The purpose is to give us the opportunity to update you all with what we are doing and give you the opportunity to have a say, ask questions you may have to the team, voice complaints, make requests and in general communicate with us directly. This Q&A will not be quite like our office hours. It will not be as formal, it will be far less structured, and it will not be streamed live.

Seating is limited to 70 avatars and is first come first serve. We will, however, try to record it and offer the recording for folks who couldn’t make it.

All are invited, fans and haters alike, people who use our viewers and those who don’t… everyone.
All questions are welcome, we only ask that language be kept PG and folks be civil to each other.

Again, note that the meeting will not be streamed live. The venue for the meeting is the Firestorm Office Hours meeting place on the Phoenix Firestorm support region.

Viewer release summary 2013: week 6

This summary is published every Monday and is a list of SL viewer / client releases (official and TPV) made during the previous week. When reading it, please note:

  • It is based on my Viewer Round-up Page, a list of  all Second Life viewers and clients that are in popular use (and of which I am aware) and which are recognised as adhering to the TPV Policy
  • By its nature, this summary will always be in arrears
  • The Viewer Round-up Page is updated as soon as I’m aware of any releases / changes to viewers & clients, and should be referred to for more up-to-date information as the week progresses
  • The Viewer Round-up Page also includes comprehensive links to download pages, blog notes, release notes, etc., as well as links to any / all reviews of specific viewers / clients made within this blog.  

Updates for the week ending: 10 February, 2013

  • SL Viewer updates:
      • Beta version updated to 3.4.5.270034 on February 6th –  release notes
      • CHUI development viewer reached release 3.4.6.270209 on February 9th
  • Kokua released version 3.4.425654 r6 on Feb 6th – core updates: Advanced Build Tools floater offers Imprduence layout and elements from Firestorm; Metropolis Metaverse added to OpenSim grid list – release notes
  • Cool VL updated on the 9th of February and again on the 10th to arrve at:
    • Stable version: 1.26.6.10
    • Legacy version (v2.6 renderer): 1.26.4.53
    • Experimental version: 1.26.7.10
    • Release notes
  • Lumiya released version 2.4.2 on February 4th – core update: mesh object suppor fixes –  release notes
  • Group Tools released version 2.2.16.0 on February 6th

Discontinued Viewers

  • Phoenix officially reached end-of-line for SL on December 31st – read more here
  • Zen viewer was withdrawn from the SL TPV directory and all repositories shutdown on January 27th, 2013.

Related Links

SL projects news week 6 (3): viewer and CHUI, SSB, materials

SL Beta Viewer and CHUI

As noted in part 2 of this report, a new beta release of the SL viewer was made on February 6th. The crash rates for this version, some 48 hours on from the release, are reported by Oz Linden as being “remarkably low”, with some 1400 users / half-a-million minutes logged against it to date. If this continues through the weekend, the chances are the code will go to a release version of the viewer early in week 7 (week commencing Monday 11th February).

If things go according to plan, there is likely to be one more 3.4.5 beta release prior to a merge taking place with the Communications Hubs User Interface (CHUI) code.

Currently, the CHUI code is now fully merged with the viewer-dev code, although it is slightly out-of-step with the beta code. Once the merge to beta has been made, and with everything else going on with the viewer, it is anticipated that CHUI will have “a nice long run,” as a beta release.

Server-side Baking and Pile-on Test Pre-announcement

Avatar bake fail
SSB: looking to solve avatar bake fail

Nyx Linden and his team continue to work on server-side baking (SBB), both on the server and the viewer side of things. An update to the project viewer and viewer code was made back on January 30th, and Nyx has indicated that another push is due early in week 7.

The major news on this project is that a pile-on test is in the planning stage. While dates and times have yet to be confirmed, this is being targeted for later in February on Aditi, and the Lab is looking to have people running a Server-side baking viewer (e.g. the project viewer or any TPV which has implemented the necessary viewer-side code) attend and help provide a thorough load test for the new code.

The test will require viewers to be enabled for certain types of data logging in order to ensure LL gather the relevant information. Details on what will be required will be passed on to those TPVs which indicate they can participate in the test. For those individuals wishing to help out, but whose preferred viewer has not enabled the SSB code by the time of the test, the LL project viewer will have the necessary logging options already enabled. Further details on the proposed test will be published here as they become available.

In terms of TPV integration, it has been confirmed that the SSB code changes for the viewer impact RLVa. Details of the overall impact is unclear, other than RLVa  is affected in several areas and Kitty Barnett is working to fix issues. However, this does mean that integration of the SSB code has slowed for some TPVs.

Materials Processing and Project Viewer

As previously reported, the server-side code is on the BlueSteel RC channel, and there are no apparent issues with the code which might prevent its wider distribution. There are still some issues to resolve with materials processing, which means that a project viewer may be delayed another week or so.

Some of the problems appear to have been network related. When testing recent versions of the pre-release materials viewer on a materials-enabled region of the Main grid, it was noted that they (quote) “significantly added” to the network load for the test region, impacting performance. These problems required additional investigation in order to resolve, and it is hoped that they have now been “wrestled to the ground”. There were also some reports of a couple of things still to be fixed in the UI, although it appears that, again, these have now been dealt with.

Given this, and while a publicly available project viewer appearing in week 7 is “not out of the question”, the Lab appear to be looking at week 8 (commencing Monday 18th February) as the week in which a materials project viewer will surface. In the meantime, the recommendation remains that TPVs do not pull code from the current materials code repository for integration into their own viewers.

The look of things to come: Materials Processing
The look of things to come: Materials Processing should have a public viewer available in the next two weeks

FMODex

Oz confirmed that the Lab now has FMODex working internally on both Windows and Linux, and passed on thanks to members of the Singularity team for their work in making this possible. The code is to be made available for wider testing by TPVs “Pretty Soon”TM.

As per my last update on the matter, it is unlikely that LL will be able to pre-build the code for TPVs, but the goal is to have something which is not “dramatically different” to how TPVs previously obtained the required code. However, with all that is happening at the moment, it is possible that those TPVs which haven’t already looked to using FMODex may continue “as is” with FMOD until some of the pressure is off in terms of other viewer projects which are rolling forward.

Related Links

Information source: TPV Dev meeting, Friday 8th February, 2013. There is unlikely to be an official audio recording of this meeting due to it being relocated due to ongoing issues with the Hippotropolis region.

SL project news week 6 (2): server deployment updates

Server Deployments – Week 6

The planned server deployments for week 6 occurred as anticipated:

  • On Tuesday 5th February, the Main channel received the server maintenance project deployed to LeTigre in week 5. This has miscellaneous minor bug fixes and new features – release notes
  • On Wednesday 6th February, the RC channels received the following:
    • BlueSteel: code for materials processing (project viewer still pending) – release notes
    • LeTigre: a new maint-server project to fix miscellaneous crash modes, and with minor performance improvements – release notes
    • Magnum: interest list code update to specifically address the bot / bandwidth problem reported on in last week’s update and also support for materials processing – release notes

Server Deployments – week 7

There is no advanced news on potential deployment for the week commencing Monday 11th February, 2013.

SL Viewer Updates

The beta viewer was updated on February 6th with the release of 3.4.5.270034. Please refer to the release notes for details of all changes and updates. The CHUI project development viewer also updated to 3.4.6.270114 on February 6th.

Updates – Issues and Other Bits

Bot / Bandwidth Issues

Speaking at the Server Beta user group meeting on Thursday February 7th, Maestro Linden indicated the ongoing bot / bandwidth issue related to the interest list code and as pointed to by Latif Khalifa and confirmed by Andrew Linden (reported in more detail here), appears to have been resolved. Commenting on the bug fix in the server deployment thread, Triple Peccable, who was one of those being badly impacted by the problem, comments:

Maestro and Andrew,

I wanted to report on the bot’s usage. Fixed!

Before this incident the bot’s “normal” usage was 5 MB / hr. That is so normal no one would suspect anything.

But now it is 1 MB / hr! It has never been that low before, ever.

The improvement might be from the interest list changes, but since the bot is parked 3300m up with a very limited draw distance, I think it is from this UDP bug fix, and will help with more than just bots. :smileyhappy:.

Estate Ban Issues

Two issues have been reported in relation to estate bans recently.

One is the use of LSL commands for estate moderation, as mentioned in the second part of my report for week 5. While it is not clear how widespread the issue is (the reports received so far appear to relate to four regions), it had been hoped that the code deployment to LeTigre might have fixed the problem, but tests with an affected region move to LeTigre showed this was not the case. However, Maestro Linden believes LL may have a match between the issue and a bug that was filed internally after  crash report fingerprints were browsed, so investigations are liable to continue.

In the second, Whirly Fizzle has reported an issue with the “GTFO” ban feature in Phoenix. While this adds the banned individual’s name to the banlist for an estate, the individual isn’t actually barred from accessing the estate. As such, it is thought that this issue might contribute to recent problems in people apparently circumventing estate bans, and is something which will not be rectified by the estate ban improvements currently being deployed by LL, as it is an issue within the Phoenix viewer code itself.

Region Crashes on Restarts

In addition to the restart performance issues related to physics memory use previously reported and updated in part 1 of this report, some regions are experiencing issues with the physics engine during a restart, with all scripting capabilities being disabled as the physics engine is overloaded. Scripting must then be re-enabled by the region owner / estate managers. A fix for this is being worked on, and should be available soon.

Vanishing Regions

Following the week 5 deployments, Alvid Majestic contacted me concerning issues with regions diagonally opposite Brocade, on the Mainland, failing to render in the viewer’s world view, and would not render until such time as a person moved into one of the regions immediately adjacent to it / moved into it.

Missing regions: Mullein and (beyond it) Ear fail to render from Brocade, which sits diagonally opposite them
Missing regions: Mullein and (beyond it) Ear fail to render from Brocade, which sits diagonally opposite them

This is not a new issue, having previously been reported in SVC-8130, although there was some confusion as to whether or not it had been resolved. Commenting on it in general at the Server Beta User Group meeting, Maestro Linden informed me, “It’s somewhat rare, but it was never officially fixed.”  As the JIRA is closed to comment, Shug Maitland has raised a forum thread on matter, so if you are witnessing the same issue on an ongoing basis, consider adding your comments there as well as raising reports.

Region Crossings

There has been mixed feedback to the results of the deployment of the new region crossing code across Agni.

Regular commentator on this blog, Wolf Baginski Bearsfoot has put together a report on his findings in the SL Server sub-forum, which builds on his initial impressions posted in this blog.

Some feedback given through the User Groups suggest that in some instances region crossings – such as with sailing – are improved, and at the Simulator User Group meeting on Tuesday 5th February, Simon Linden indicated LL were seeing fewer instances of stuck teleports.However, there have also been reports passed through the Server Beta group of automated cars on the Mainland encountering problems at region crossings while following Linden Roads and piling-up at the boundaries of regions such as Furness to Ravenglass, although instances appear to have calmed down. More updates on this as they come.