SL project updates 44/2: TPV Developer meeting

The updated official viewer splash screen for general users (i.e. not first-time log-in)

The following notes are taken from the TPV Developer meeting held on Friday, November 3rd 2017. The video of that meeting is embedded at the end of this update, my thanks as always to North for recording and providing it.

SL Viewer

The Voice RC viewer has one remaining significant bug, which causes some of those on the Windows version of the viewer to connect to the wrong voice channel, which is preventing this viewer progressing further.

The 64-bit Alex Ivy RC viewer updated to version 5.1.0.510354 on November 2nd, 2017. Overall, this viewer is described as doing “really well”, although there are still crash issues with it. These are most noticeable with people on 32-bit Windows systems (those on 64-bit versions of Windows running the 64-bit version of the viewer are experiencing far fewer crashes).

The latest update should correct issues with the viewer’s updater, and it is likely more users will be added to the RC cohort usage pool before this viewer is promoted to release status.

Work is once again proceeding with the 360-snapshot viewer, with improvements to image quality and processing speed, and a new update should be appearing soon. The Lab is also working on a new means to upload 360 snapshots from the viewer to SL Place pages.

Inventory UDP Messaging

Work has started in deprecating all UDP inventory messaging. This is not progressing “super fast”, as it is being progressed alongside other work, and the projected end date is some time “fairly soon” after the end-of-year holidays, when back-end support for the UDP messaging will be turned off. This means that any active viewers still using the UDP inventory handling routes should be making the move to HTTP.

No Change Windows

With the end-of-year holiday season approaching, the Lab is looking at dates for no change windows – periods when they will not be making and simulator or viewer releases, and would prefer to see TPVs do the same.

The first of these periods will be the US Thanksgiving holiday period, when a no change period is liable to be enforced from Wednesday, November 22nd, with the all-clear on Monday, November 27th (subject to formal confirmation).  The Christmas no change window is still TBD, but will likely be from at least Friday, December 22nd through until shortly after the new year.

Other Items

Resource Usage tools

Chalice Yao has proposed a feature to the Firestorm team to allow users better understand the resources they are using, both through their avatar’s VRAM usage and the VRAM, triangles and vertices for any selected object (see FIRE-21793). As the Lab is currently working on amending how rendering cost calculations, a more detailed discussion on these ideas has been tabled for the next TPV Developer meeting, on Friday, November 17th.

Firestorm Release

The next Firestorm release has been delayed of late, but recently entered beta testing, with the aim of it appearing before the end of the year. When it arrives, I’ll have me usual overview of significant updates, but Beq Janus recently blogged about a couple of updates she has contributed to the release, and the Lab have indicated their own interest in possibly adopting the updates, if contributed.

Second Life Minimum System Requirements

It has been noted that the specified minimum system requirements for Second life may be out-of-date (see BUG-139301), and this may be exacerbated with the Alex Ivy viewer. It’s likely that the specifications will be looked at again.

Advertisements

SL project updates 40/2: TPVD Meeting; e-mail verification

Mitsumi-Town in Tokyo; Inara Pey, September 2017, on FlickrMitsumi-Town in Tokyoblog post

The majority of these notes are taken from the TPV Developer meeting held on Friday, October 6th 2017. The video of that meeting is embedded at the end of this update, my thanks as always to North for recording and providing it. time stamps in the text below will open the video in a separate tab for ease for reference to the relevant points of discussion in the meeting.

The meeting was chaired by Rider Linden, who was filling in for Oz, and was somewhat foreshortened.

SL Viewer

There were no updates to the current crop of official viewers during the week, leaving the pipeline as:

  • Current Release version 5.0.7.328060, dated August 9th, promoted August 23rd – formerly the Maintenance RC
  • Release channel cohorts:
    • Voice RC viewer, version 5.0.8.329250 on Friday, September 29th
    • Maintenance RC viewer, version  5.0.8.329115, dated September 22nd.
    • Wolfpack RC viewer,version  5.0.8.329128, dated September 22nd – this viewer is functionally identical to the release viewer, but includes additional back-end logging “to help catch some squirrelly issues”
    • Alex Ivy 64-bit viewer, version 5.1.0.508209, dated September 1st
  • Project viewers:
  • Obsolete platform viewer version 3.7.28.300847, dated May 8th, 2015 – provided for users on Windows XP and OS X versions below 10.7.

Quick notes:

  • [4:35-4:52] Work is continuing to fix the texture crashes in the 64-bit Alex Ivy RC.
  • [5:25-5:40] There is a reference to Voice changes, however, the question is non-specific, and it is unclear if it refers to the current Voice RC (updated at the end of week #39) or to further updates, the question is deferred to Oz (who is directly involved in the Voice project).

E-mail Verification

[2:56-4:00] As noted in part 1 of this week’s report, the Main channel deployment saw an update to IMs-to-e-mail as part of the Lab’s move to only handling verified e-mail addresses.

The sending / forwarding of messages to unverified addresses is not blocked – as yet. However, once a viewer-side changed has been deployed, accounts with unverified emails will no longer be able to request that IMs be forwarded to e-mail; attempts to enable the viewer-side setting will fail, and result in a message advising the account holder to verify their e-mail address.

Potential Interest List Change

[15:52-18:40] (with a lengthy silence)] There is a potential simhost change to the Interest List under discussion at the Lab. This will in essence reduce some of the aggressive culling of updates from objects behind an avatar (particularly useful in the case of light sources, moving objects, etc). If it goes ahead, this update could also benefit the 360-snapshot viewer.

Other Items

 

Estate Tools Update

[6:07-10:40 (ish) and 13:10-14:20] There is a lengthy on-off exchange, in Voice and text (with some long pauses in both) regarding the Estate Tools updates in the viewer. In short: updates to the estate ban lists (see BUG-8883 and BUG-40676 as examples of requested changes), have resumed. however, as Alexa Linden, who is running this project, was unavailable for the meeting, no detailed update was possible.

Viewer Build Issue

[10:50-12:00] There is “progress” in fixing an incorrect viewer build update involving a change to the Cmake files which results in a forced rebuild of the Windows version of the viewer (the “-z0” issue).

In-world Posing

[20:48-end of meeting] NiranV Dean has been experimenting with a means of in-world posing an avatar through the viewer (e.g. for things like photography), which would be visible to the person posing their avatar (as it would require server-side support to be visible in all viewers). There may be interest from the Lab in accepting as a proposal / contribution, if the details of what is required can be agreed between the Lab and Niran.

There were concerns among those attending the meeting (and raised largely in text) on what Niran is trying to achieve and how it might related to content ripping. If nothing else, the conversation underlines the need for a clearly thought-out proposal outlining the idea, its purpose, how it might be implemented, potential issues which will need to be considered and addressed, etc., rather than requesting people “try” the idea and give feedback.

Date of Next Meeting

The next Third-Party Viewer Developer meeting will not be until Friday, November 3rd, 2017.

 

SL project updates week 38/3: TPV Dev meeting and the Cloud

Gentle Breezes; Inara Pey, September 2017, on FlickrGentle Breezesblog post

Updated, September 25th: As indicated to me by Grumpity Linden, the cause for the Wolfpack and Maintenance RCs to be withdrawn as noted in this article (and which was as a result of this issue), has now been resolved and the two updated versions of these viewers are once again available. As a result, the links to their release notes and download options have been restored.

The majority of the notes in this update are taken from the TPV Developer meeting held on Friday, September 22nd 2017. The video of that meeting is embedded at the end of this update, my thanks as always to North for recording and providing it. Timestamps in the text below will open the video in a separate window at the relevant point for those wishing to listen to the discussions.

Server Deployments Week #38 – Recap

  • There was no deployment / restart on the Main (SLS) channel on Tuesday, September 18th, leaving that channel running on 17#17.09.01.508236.
  • On Wednesday, September 19th, the RC channels were updated as follows:
    • BlueSteel and LeTigre received a new server maintenance package, 17#17.09.14.508549, comprising improvements to address some problems that could degrade simulator performance in rare cases.
    • Magnum received a new server maintenance package, 17#17.09.14.508533, containing a fix for BUG-100505  “llGetEnv (“agent_limit”) is returning an empty string in Magnum, LeTigre and Blue Steel regions.”

SL Viewers

Alex Ivy 64-Bit

[0:54 and 6:00] The Alex Ivy 64-bit viewer is due an update, possibly in the early part of week #39 (commencing Monday, September 25th). This may not have all the fixes required for the viewer to get promoted to de facto release status. Before this happens, the Lab wants to tackle the problem with pipeline stalls in this viewer, and are working on an experimental branch of the viewer to try to resolve the issue. This branch will be made available as a test viewer to those who have reported the issue and can reliably repro it. Depending on the outcome of this testing, a decision will be made on folding it into the RC branch for the viewer.

The wiki instructions for the viewer should now be updated to the 64-bit build requirements, nd Oz indicates that a new 64-bit Havok library should follow the release of the viewer.

Voice Viewer

[1:33, 2:13, and 34:02-37:14] There will be a new Voice SDK arriving for the Voice RC viewer in the near future, which will include an updated SDK that includes a fix for some long-standing problems. There are still some problems to be fixed, so it is unlikely this viewer will be promoted until the new SDK has spent time in RC and the remaining major issues have been resolved.

This viewer already fixes the high number of failures to connect to the Voice service when logging-in; however, there is an issue where manually killing the Voice process will not restart (as it used to), and so Voice won’t work. The Lab would like to fix this so the process does restart the process, but this is not seen as a critical issue to be resolved before the viewer is promoted.

The new SDK does not alter the Voice protocols, but is not compatible with previous versions, requiring the supporting updates in the viewer to work. This means the new SDK cannot work with older viewer versions, and older SDKs cannot be used with viewers incorporating the code updates to support this new SDK.

Maintenance and Wolfpack RCs

[2:04 and 4:15] The meeting references updates to the Maintenance RC viewer (to 5.0.8.329115) and the Wolfpack RC (to 5.0.8.39128). While both updates were available at the time of the meeting and shortly thereafter, the Alternate Viewers wiki page now references the previous RC releases for both (5.0.8.329065 and 5.0.8.328990 respectively). It is not clear whether this is an error with the wiki page, or if the updated RCs have been withdrawn (both still appear on the viewer release notes list). Resolved.

Snapshot Viewer

[5:14] There may be a new updated to the 360 snapshot viewer in the next week to two weeks. Work has also started on providing better support for using 360-degree images in Second Life Place Pages (see here and here for more on Place Pages).

Pipeline Status

Keeping the above in mind, the current viewer pipeline comprises:

  • Current Release version 5.0.7.328060, dated August 9th, promoted August 23rd – formerly the Maintenance RC
  • Release channel cohorts:
    • Maintenance RC viewer, version 5.0.8.329065, dated September 18th.
    • Wolfpack RC viewer,version 5.0.8.328990, dated September 12th – this viewer is functionally identical to the release viewer, but includes additional back-end logging “to help catch some squirrelly issues”
    • Alex Ivy 64-bit viewer, version 5.1.0.508209, dated September 5th
    • Voice RC viewer, version 5.0.8.328552, dated September 1st
  • Project viewers:
  • Obsolete platform viewer version 3.7.28.300847, dated May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

New Viewer Splash / Log-in Screen

[7:12-7:42] As noted in my week #36 TPV meeting notes, Phronimos Linden is updating the viewer splash screen which will see a different look and feel to the screen, including how information is displayed (such as making grid status info more prominent), and will see updates to some of the widgets providing information in the splash screen. This work is now with the QA team, and information on the updates will be available for TPVs soon.

Windows Viewer  Installation Warning

[7:47-8:48] The Lab’s code-signing key used to verify the viewer with Windows (notably Windows 10) has expired. The Lab have a new key, but for an interim period, it means users installing the Windows version of the official viewer may find Windows SmartScreen reports the viewer as unverified.

You can read more here, on via my own blog post, which includes steps on clearing the warning and installing the viewer.

Server Version Updates and Move to the Cloud

[12:13-12:53] A number of server version updates are advancing. these don’t always have user-visible changes, but they are nevertheless important to Second Life. Among other things, they are part of the preparatory work for moving SL capabilities to the cloud (see my week #36 TPV notes for more on this).

[15:08-19:06] There is no time line for moving things to the cloud, simply because the Lab does not know at this point how long it will take. There are some significant changes which must be made to both the way things are built and the way they are run, and there need to be assorted updates to various components that go into building and running SL services.

Some SL services are already being tested in the cloud, and some are performing well – such as the process for determining if a user requires a viewer update. Others have been tested and revealed problems which must be addressed if they are to be run from the cloud – or should be addressed even when not running in the cloud.

It is unlikely the Lab will be providing specifics on services which have moved to the cloud / are being tested, and which are still based within their data centre until things reach a point where simulators are running in the cloud, simply because where many SL services run makes absolutely no difference to the user experience, as long as they are running. Moving and testing simulators in the cloud is likely to be one of the last things to be tackled, simply because of the complexities involved.

The first goal is to get everything working pretty much “as is” from the cloud. Only after this has been done, will work start on leveraging the benefits of having everything in the cloud be explored and exploited.

[19:06-21:49] This could include giving – and to use Oz’s words, the option of having their regions hosted in specific geographical locations. So, for example, the various communities located in South America could have their regions all hosted in South America, potentially improving response times between viewer and server.  However, whether this will in fact be possible is dependent on the Lab reaching that point at which they can start leveraging the benefits of the cloud.

Obviously there are trade-offs in this kind of shift, should it occur; relocating a simulator to better serve a community may not improve things for others access the region on that simulator. However, in potentially supplying the option, the Lab is providing land owners with a choice of what they would like to do.

[21:56-22:38] If nothing else, this work should be a demonstration that the Lab really is continuing to invest in Second Life and its future. Were they seriously thinking of letting it go (i.e. in favour of Sansar), then none of this work  – and the associated expenditure – would be taking place.

Environment Enhancement Project (EEP)

(See also my week #38 CCUG update.)

[29:26-31:16] There is a “fair amount” of back-end work that is being worked through, and the work is approaching the point of internal testing within the Lab. Once this has reached a suitable point, the server-side / simulator changes will be deployed (e.g. to Aditi) for wider testing, alongside of a project viewer to handled the client-side application of the capabilities.

Recent Grid Issues

[37:51-39:20] As most are aware, there have been some recent grid issues. While not the cause of these issues, but which has been a contributing factor to their duration, is some low-level code within the viewer which handles log-in retries far too aggressively. When this happens en masse (such as when there is a grid issue), it results in the log-in servers being swamped, adding to the woes for people trying to log-in.

A recent Maintenance update to the SL viewer addresses this issue (see my week #30 TPV update), and the request for TPVs to pick these code changes up was re-iterated at the meeting. In addition, the log-in servers have themselves been made more robust when facing large number of attempted / repeated log-in attempts.

Other Items

Estate Tool Ban List Improvements

[9:32-11:22] The Lab has resumed work on the region ban lists (layout / usability, etc), and the updates should be appearing soonTM. The specifics of what is being done will hopefully be available for the next TPV Developer meeting.

Premium Member Benefits

[13:13-14:53] There is apparently at least one Premium member benefit that will be appearing  real soonTM which the Lab believe people will like, and some further ideas are being considered.  Oz declined to comment on what any of these might be, citing it being more fun to find out when they are announced. He also indicated that appropriate and considered suggestions  / ideas for benefits (e.g. not things that persist after a Premium subscription has been cancelled) are also welcome.

Group Notice Failures

[28:00-28:55] Still no work on group notices (on-line and off-line) sometimes not getting through for some people. It’s not on the “now / next” roadmap of things the Lab is / will be looking at. The focus on sever-side work is on dealing with instability issues which can cause crashes / offer exploits to griefers.

Asset HTTP Messaging and Asset HTTP Issues

[41:14] As noted in my week #36 TPV meeting update, the recent Asset HTTP updates are leading to the texture pipeline getting out of sync, and people experiencing texture load stalls. A JIRA for this has been filed (BUG-139123), and a possible fix has been submitted to the Lab by Sovereign Engineer.

[43:16] The Lab is also working on the texture caches in an attempt to make them faster and more effective.

 

 

SL project updates week 36/2: TPV Dev meeting + SL in the cloud

Brand New Colony; Inara Pey, September 2017, on FlickrBrand New Colonyblog post

Updated to reflect the release of the Wolfpack RC, which appeared after this article had been uploaded for publishing.

The majority of the notes in this update are taken from the TPV Developer meeting held on Friday, September 8th 2017. The video of that meeting is embedded at the end of this update, my thanks as always to North for recording and providing it. Timestamps in the text below will open the video in a separate window at the relevant point for those wishing to listen to the discussions.

Server Deployments Week #36 – Recap

Please refer to the deployment notice for the week for latest updates and news.

Region Return Issues

Whether connected to the Wednesday deployment or the grid-wide issues experienced later that day, but some regions reported significant returns of in-world objects (Mother Road, on the Main (SLS) channel, which I blogged about here, suffered the problem, as Heavenly Views (Magnum RC) also apparently did). Most of these problems seem to have been rectified by a roll-back of the affected regions.

SL Viewer

A new RC viewer, codenamed Wolfpack – version 5.0.8.328879 – was released late on September 8th. This viewer is functionally identical to the release viewer, but includes additional back-end logging to “help catch some squirrelly issues”.

[1:15-4:15] Otherwise have been no further viewer updates since the start of the week, leaving the current pipeline as follows:

  • Current Release version 5.0.7.328060, dated August 9, promoted August 23 – formerly the Maintenance RC
  • RC viewers:
    • Alex Ivy 64-bit RC viewer version 5.1.0.508209, dated September 5th
    • Voice RC viewer updated version 5.0.8.328552, dated September 1st
    • Maintenance RC viewer version 5.0.8.328812, dated August 31st
  • Project viewers:
  • Obsolete platform viewer version 3.7.28.300847, dated May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

The Alex Ivy and Voice updates bring these RCs into parity with the current release viewer. These updates have lowered the overall crash rates for both viewers, although the Voice viewer’s crash rate is still somewhat elevated compared to the release viewer. This elevated crash rate is something the Lab has seen in the various viewers for the last few months, and are working to lower.

Alex Ivy will likely have two more RC releases before it is ready for promotion to de facto release status, as there are a couple of issues the Lab wants to clear up before any promotion of this viewer.

It is hoped work will shortly be resuming on the 360-degree snapshot viewer, part of which will also be bringing it up to parity with more recent releases.

New Viewer Splash / Log-in Screen

[4:25-8:07] The Lab is in the process of revamping the official viewer’s splash / log-in screen. This should be inherited automatically by those TPVs using the Lab’s splash screen. This will see the screen have a different look and feel, with the information arranged a little differently, with the grid status made somewhat more prominent. Some of the information widgets associated with the splash screen are also being updated as a part of this work, and will require testing by TPVs when available (RSS feeds should not be affected).

This work is being carried out by Phronimos Linden, one of the more recent (3 months ago at the time of writing) Lab staff recruited to work on Second Life.

Moving to the Cloud

[15:52-25:25] As confirmed in a recent Lab blog post, Second Life services are moving to the cloud. There are no specific details on this as yet, but in short:

  • The work will be carried out in stages.
  • It will include providing regions from the cloud (rather than the Lab’s own co-located servers).
  • As many know, the asset services has been in S3 cloud servers for several years.
  • The order of moving services hasn’t been determined as yet, but the aim will be to move services, then test for reliability and performance before opening them up / moving to the next service(s).
  • Some testing has already begun, using services only accessed indirectly by users.
  • The Lab has been working on the infrastructure for this for a while, but none of the major services have been moved.

It is unlikely this will lead to regions of a different size to those presently on the grid, or which can be varied in size (a-la the OpenSim varregions), as region size is very deeply baked into the simulator and viewer code (and probably elsewhere in the overall SL code base); although some at the Lab would love to be able to have such a region capability. However – as indicated in the Lab’s blog post – it may allow the Lab to introduce new region products, possibly ones capable of handling greater loads.

It is hoped that this work will result in lower tier over time, but whether this will be the case of not is still and open question at this time, as it is unclear as to what costs will be involved in terms of cloud instance types., etc., that will be required to support simulators.

Overall, there is a lot the Lab hopes to achieve with the move, but the precise benefits are likely to only become clear once things have been tried and found to work / over time as the work progresses and beyond. However, at this point time scales are TBD, and it is liable to be some time before user-visible aspects of the service move to the cloud (although non-visible services – such as the log-in service, for example – could be moved sooner).

A precursor to this project is the continuing work to update the servers to newer version of Linux, work which is making “good progress”.

Other Items

Asset HTTP Messaging and Asset HTTP Issues

[10:11-10:45 and 12:10-15:23] With the promotion of the Asset HTTP viewer to release status in June 2017, the majority of assets are now delivered to the viewer using HTTP via the Content Delivery Network(s) used by the Lab. This means that UDP messaging for all of the affected asset types will be turned off at the server end at some point.  Currently no date has been set for this, and it looks like it may not occur much before February 2018.

There is still an issue related to fetching assets over HTTP in general which is experienced by some users some of the time   whereby something causes the pipeline textures to get out of sync and things to go awry. As the problem happens for some and not others, the Lab is still trying to determine the root cause for the problem, but it is a matter the Lab is trying to resolve.

Catznip viewer reports issues with textures over HTTP “stalling” for between 5-10 minutes on their pre-release viewer with the latest HTTP updates, but this isn’t something the Lab is aware of as a more general issue.

Estate Tool Ban List Improvements

[10:48-11:43] The Lab did some initial work to improve ban lists at the estate / region level a while ago, but the intended work to improvement to overall layout for the lists, etc., has been delayed do to work on dealing with viewer crashes, etc. This work is apparently now “next in line” once some of the existing viewer projects are shipped.

The conversation from 26 minutes to the end is more general chat on viewer stats, speculation on the reason for the texture load stalls, and an ad for the Firestorm birthday party.