SL project updates week 38/3: TPV Dev meeting and the Cloud

Gentle Breezes; Inara Pey, September 2017, on FlickrGentle Breezes – blog post

Updated, September 25th: As indicated to me by Grumpity Linden, the cause for the Wolfpack and Maintenance RCs to be withdrawn as noted in this article (and which was as a result of this issue), has now been resolved and the two updated versions of these viewers are once again available. As a result, the links to their release notes and download options have been restored.

The majority of the notes in this update are taken from the TPV Developer meeting held on Friday, September 22nd 2017. The video of that meeting is embedded at the end of this update, my thanks as always to North for recording and providing it. Timestamps in the text below will open the video in a separate window at the relevant point for those wishing to listen to the discussions.

Server Deployments Week #38 – Recap

  • There was no deployment / restart on the Main (SLS) channel on Tuesday, September 18th, leaving that channel running on 17#17.09.01.508236.
  • On Wednesday, September 19th, the RC channels were updated as follows:
    • BlueSteel and LeTigre received a new server maintenance package, 17#17.09.14.508549, comprising improvements to address some problems that could degrade simulator performance in rare cases.
    • Magnum received a new server maintenance package, 17#17.09.14.508533, containing a fix for BUG-100505  “llGetEnv (“agent_limit”) is returning an empty string in Magnum, LeTigre and Blue Steel regions.”

SL Viewers

Alex Ivy 64-Bit

[0:54 and 6:00] The Alex Ivy 64-bit viewer is due an update, possibly in the early part of week #39 (commencing Monday, September 25th). This may not have all the fixes required for the viewer to get promoted to de facto release status. Before this happens, the Lab wants to tackle the problem with pipeline stalls in this viewer, and are working on an experimental branch of the viewer to try to resolve the issue. This branch will be made available as a test viewer to those who have reported the issue and can reliably repro it. Depending on the outcome of this testing, a decision will be made on folding it into the RC branch for the viewer.

The wiki instructions for the viewer should now be updated to the 64-bit build requirements, nd Oz indicates that a new 64-bit Havok library should follow the release of the viewer.

Voice Viewer

[1:33, 2:13, and 34:02-37:14] There will be a new Voice SDK arriving for the Voice RC viewer in the near future, which will include an updated SDK that includes a fix for some long-standing problems. There are still some problems to be fixed, so it is unlikely this viewer will be promoted until the new SDK has spent time in RC and the remaining major issues have been resolved.

This viewer already fixes the high number of failures to connect to the Voice service when logging-in; however, there is an issue where manually killing the Voice process will not restart (as it used to), and so Voice won’t work. The Lab would like to fix this so the process does restart the process, but this is not seen as a critical issue to be resolved before the viewer is promoted.

The new SDK does not alter the Voice protocols, but is not compatible with previous versions, requiring the supporting updates in the viewer to work. This means the new SDK cannot work with older viewer versions, and older SDKs cannot be used with viewers incorporating the code updates to support this new SDK.

Maintenance and Wolfpack RCs

[2:04 and 4:15] The meeting references updates to the Maintenance RC viewer (to 5.0.8.329115) and the Wolfpack RC (to 5.0.8.39128). While both updates were available at the time of the meeting and shortly thereafter, the Alternate Viewers wiki page now references the previous RC releases for both (5.0.8.329065 and 5.0.8.328990 respectively). It is not clear whether this is an error with the wiki page, or if the updated RCs have been withdrawn (both still appear on the viewer release notes list). Resolved.

Snapshot Viewer

[5:14] There may be a new updated to the 360 snapshot viewer in the next week to two weeks. Work has also started on providing better support for using 360-degree images in Second Life Place Pages (see here and here for more on Place Pages).

Pipeline Status

Keeping the above in mind, the current viewer pipeline comprises:

  • Current Release version 5.0.7.328060, dated August 9th, promoted August 23rd – formerly the Maintenance RC
  • Release channel cohorts:
    • Maintenance RC viewer, version 5.0.8.329065, dated September 18th.
    • Wolfpack RC viewer,version 5.0.8.328990, dated September 12th – this viewer is functionally identical to the release viewer, but includes additional back-end logging “to help catch some squirrelly issues”
    • Alex Ivy 64-bit viewer, version 5.1.0.508209, dated September 5th
    • Voice RC viewer, version 5.0.8.328552, dated September 1st
  • Project viewers:
  • Obsolete platform viewer version 3.7.28.300847, dated May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

New Viewer Splash / Log-in Screen

[7:12-7:42] As noted in my week #36 TPV meeting notes, Phronimos Linden is updating the viewer splash screen which will see a different look and feel to the screen, including how information is displayed (such as making grid status info more prominent), and will see updates to some of the widgets providing information in the splash screen. This work is now with the QA team, and information on the updates will be available for TPVs soon.

Windows Viewer  Installation Warning

[7:47-8:48] The Lab’s code-signing key used to verify the viewer with Windows (notably Windows 10) has expired. The Lab have a new key, but for an interim period, it means users installing the Windows version of the official viewer may find Windows SmartScreen reports the viewer as unverified.

You can read more here, on via my own blog post, which includes steps on clearing the warning and installing the viewer.

Server Version Updates and Move to the Cloud

[12:13-12:53] A number of server version updates are advancing. these don’t always have user-visible changes, but they are nevertheless important to Second Life. Among other things, they are part of the preparatory work for moving SL capabilities to the cloud (see my week #36 TPV notes for more on this).

[15:08-19:06] There is no time line for moving things to the cloud, simply because the Lab does not know at this point how long it will take. There are some significant changes which must be made to both the way things are built and the way they are run, and there need to be assorted updates to various components that go into building and running SL services.

Some SL services are already being tested in the cloud, and some are performing well – such as the process for determining if a user requires a viewer update. Others have been tested and revealed problems which must be addressed if they are to be run from the cloud – or should be addressed even when not running in the cloud.

It is unlikely the Lab will be providing specifics on services which have moved to the cloud / are being tested, and which are still based within their data centre until things reach a point where simulators are running in the cloud, simply because where many SL services run makes absolutely no difference to the user experience, as long as they are running. Moving and testing simulators in the cloud is likely to be one of the last things to be tackled, simply because of the complexities involved.

The first goal is to get everything working pretty much “as is” from the cloud. Only after this has been done, will work start on leveraging the benefits of having everything in the cloud be explored and exploited.

[19:06-21:49] This could include giving – and to use Oz’s words, the option of having their regions hosted in specific geographical locations. So, for example, the various communities located in South America could have their regions all hosted in South America, potentially improving response times between viewer and server.  However, whether this will in fact be possible is dependent on the Lab reaching that point at which they can start leveraging the benefits of the cloud.

Obviously there are trade-offs in this kind of shift, should it occur; relocating a simulator to better serve a community may not improve things for others access the region on that simulator. However, in potentially supplying the option, the Lab is providing land owners with a choice of what they would like to do.

[21:56-22:38] If nothing else, this work should be a demonstration that the Lab really is continuing to invest in Second Life and its future. Were they seriously thinking of letting it go (i.e. in favour of Sansar), then none of this work  – and the associated expenditure – would be taking place.

Environment Enhancement Project (EEP)

(See also my week #38 CCUG update.)

[29:26-31:16] There is a “fair amount” of back-end work that is being worked through, and the work is approaching the point of internal testing within the Lab. Once this has reached a suitable point, the server-side / simulator changes will be deployed (e.g. to Aditi) for wider testing, alongside of a project viewer to handled the client-side application of the capabilities.

Recent Grid Issues

[37:51-39:20] As most are aware, there have been some recent grid issues. While not the cause of these issues, but which has been a contributing factor to their duration, is some low-level code within the viewer which handles log-in retries far too aggressively. When this happens en masse (such as when there is a grid issue), it results in the log-in servers being swamped, adding to the woes for people trying to log-in.

A recent Maintenance update to the SL viewer addresses this issue (see my week #30 TPV update), and the request for TPVs to pick these code changes up was re-iterated at the meeting. In addition, the log-in servers have themselves been made more robust when facing large number of attempted / repeated log-in attempts.

Other Items

Estate Tool Ban List Improvements

[9:32-11:22] The Lab has resumed work on the region ban lists (layout / usability, etc), and the updates should be appearing soonTM. The specifics of what is being done will hopefully be available for the next TPV Developer meeting.

Premium Member Benefits

[13:13-14:53] There is apparently at least one Premium member benefit that will be appearing  real soonTM which the Lab believe people will like, and some further ideas are being considered.  Oz declined to comment on what any of these might be, citing it being more fun to find out when they are announced. He also indicated that appropriate and considered suggestions  / ideas for benefits (e.g. not things that persist after a Premium subscription has been cancelled) are also welcome.

Group Notice Failures

[28:00-28:55] Still no work on group notices (on-line and off-line) sometimes not getting through for some people. It’s not on the “now / next” roadmap of things the Lab is / will be looking at. The focus on sever-side work is on dealing with instability issues which can cause crashes / offer exploits to griefers.

Asset HTTP Messaging and Asset HTTP Issues

[41:14] As noted in my week #36 TPV meeting update, the recent Asset HTTP updates are leading to the texture pipeline getting out of sync, and people experiencing texture load stalls. A JIRA for this has been filed (BUG-139123), and a possible fix has been submitted to the Lab by Sovereign Engineer.

[43:16] The Lab is also working on the texture caches in an attempt to make them faster and more effective.

 

 

SL project updates week 36/2: TPV Dev meeting + SL in the cloud

Brand New Colony; Inara Pey, September 2017, on FlickrBrand New Colony – blog post

Updated to reflect the release of the Wolfpack RC, which appeared after this article had been uploaded for publishing.

The majority of the notes in this update are taken from the TPV Developer meeting held on Friday, September 8th 2017. The video of that meeting is embedded at the end of this update, my thanks as always to North for recording and providing it. Timestamps in the text below will open the video in a separate window at the relevant point for those wishing to listen to the discussions.

Server Deployments Week #36 – Recap

Please refer to the deployment notice for the week for latest updates and news.

Region Return Issues

Whether connected to the Wednesday deployment or the grid-wide issues experienced later that day, but some regions reported significant returns of in-world objects (Mother Road, on the Main (SLS) channel, which I blogged about here, suffered the problem, as Heavenly Views (Magnum RC) also apparently did). Most of these problems seem to have been rectified by a roll-back of the affected regions.

SL Viewer

A new RC viewer, codenamed Wolfpack – version 5.0.8.328879 – was released late on September 8th. This viewer is functionally identical to the release viewer, but includes additional back-end logging to “help catch some squirrelly issues”.

[1:15-4:15] Otherwise have been no further viewer updates since the start of the week, leaving the current pipeline as follows:

  • Current Release version 5.0.7.328060, dated August 9, promoted August 23 – formerly the Maintenance RC
  • RC viewers:
    • Alex Ivy 64-bit RC viewer version 5.1.0.508209, dated September 5th
    • Voice RC viewer updated version 5.0.8.328552, dated September 1st
    • Maintenance RC viewer version 5.0.8.328812, dated August 31st
  • Project viewers:
  • Obsolete platform viewer version 3.7.28.300847, dated May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

The Alex Ivy and Voice updates bring these RCs into parity with the current release viewer. These updates have lowered the overall crash rates for both viewers, although the Voice viewer’s crash rate is still somewhat elevated compared to the release viewer. This elevated crash rate is something the Lab has seen in the various viewers for the last few months, and are working to lower.

Alex Ivy will likely have two more RC releases before it is ready for promotion to de facto release status, as there are a couple of issues the Lab wants to clear up before any promotion of this viewer.

It is hoped work will shortly be resuming on the 360-degree snapshot viewer, part of which will also be bringing it up to parity with more recent releases.

New Viewer Splash / Log-in Screen

[4:25-8:07] The Lab is in the process of revamping the official viewer’s splash / log-in screen. This should be inherited automatically by those TPVs using the Lab’s splash screen. This will see the screen have a different look and feel, with the information arranged a little differently, with the grid status made somewhat more prominent. Some of the information widgets associated with the splash screen are also being updated as a part of this work, and will require testing by TPVs when available (RSS feeds should not be affected).

This work is being carried out by Phronimos Linden, one of the more recent (3 months ago at the time of writing) Lab staff recruited to work on Second Life.

Moving to the Cloud

[15:52-25:25] As confirmed in a recent Lab blog post, Second Life services are moving to the cloud. There are no specific details on this as yet, but in short:

  • The work will be carried out in stages.
  • It will include providing regions from the cloud (rather than the Lab’s own co-located servers).
  • As many know, the asset services has been in S3 cloud servers for several years.
  • The order of moving services hasn’t been determined as yet, but the aim will be to move services, then test for reliability and performance before opening them up / moving to the next service(s).
  • Some testing has already begun, using services only accessed indirectly by users.
  • The Lab has been working on the infrastructure for this for a while, but none of the major services have been moved.

It is unlikely this will lead to regions of a different size to those presently on the grid, or which can be varied in size (a-la the OpenSim varregions), as region size is very deeply baked into the simulator and viewer code (and probably elsewhere in the overall SL code base); although some at the Lab would love to be able to have such a region capability. However – as indicated in the Lab’s blog post – it may allow the Lab to introduce new region products, possibly ones capable of handling greater loads.

It is hoped that this work will result in lower tier over time, but whether this will be the case of not is still and open question at this time, as it is unclear as to what costs will be involved in terms of cloud instance types., etc., that will be required to support simulators.

Overall, there is a lot the Lab hopes to achieve with the move, but the precise benefits are likely to only become clear once things have been tried and found to work / over time as the work progresses and beyond. However, at this point time scales are TBD, and it is liable to be some time before user-visible aspects of the service move to the cloud (although non-visible services – such as the log-in service, for example – could be moved sooner).

A precursor to this project is the continuing work to update the servers to newer version of Linux, work which is making “good progress”.

Other Items

Asset HTTP Messaging and Asset HTTP Issues

[10:11-10:45 and 12:10-15:23] With the promotion of the Asset HTTP viewer to release status in June 2017, the majority of assets are now delivered to the viewer using HTTP via the Content Delivery Network(s) used by the Lab. This means that UDP messaging for all of the affected asset types will be turned off at the server end at some point.  Currently no date has been set for this, and it looks like it may not occur much before February 2018.

There is still an issue related to fetching assets over HTTP in general which is experienced by some users some of the time   whereby something causes the pipeline textures to get out of sync and things to go awry. As the problem happens for some and not others, the Lab is still trying to determine the root cause for the problem, but it is a matter the Lab is trying to resolve.

Catznip viewer reports issues with textures over HTTP “stalling” for between 5-10 minutes on their pre-release viewer with the latest HTTP updates, but this isn’t something the Lab is aware of as a more general issue.

Estate Tool Ban List Improvements

[10:48-11:43] The Lab did some initial work to improve ban lists at the estate / region level a while ago, but the intended work to improvement to overall layout for the lists, etc., has been delayed do to work on dealing with viewer crashes, etc. This work is apparently now “next in line” once some of the existing viewer projects are shipped.

The conversation from 26 minutes to the end is more general chat on viewer stats, speculation on the reason for the texture load stalls, and an ad for the Firestorm birthday party.

SL project updates week 34/2: TPV Developer Meeting

Yamagata; Inara Pey, August 2017, on Flickr Yamagata – blog post

The majority of the notes in this update are taken from the TPV Developer meeting held on Friday, August  25th 2017. The video of that meeting is embedded at the end of this update, my thanks as always to North for recording and providing it. Timestamps in the text below will open the video in a separate window at the relevant point for those wishing to listen to the discussions.

Server Deployments Week #34 – Recap

Please refer to the deployment notice for the week for latest updates and news.

  • On Tuesday, August 22nd, the  Main (SLS) channel was updated with server maintenance package, 17#17.08.11.328159, comprising internal fixes and the following feature requests:
    • BUG-5398: llGetObjectDetails() constants OBJECT_SELECTED & OBJECT_SAT_UPON. This sees the addition of two new parameters:
      • OBJECT_SELECTION_COUNT – returns how many agents are selecting any link in a linkset
      • OBJECT_SITTER_COUNT – returns how many agents are sitting on any links in a linkset.
    • BUG-9666: llGetObjectDetails() constants OBJECT_REZ_TIME, OBJECT_CREATION_TIME and OBJECT_RETURN_TIME.
    • BUG-134057 OBJECT_CREATION_TIME output precision possibly clamped – this sees a shift to 6-digit precision.
  • On Wednesday, August 23rd, the three RC channels all received a new server maintenance package, 17#17.08.11.328152 comprising the MIME type changes for HTTP.

SL Viewer

[2:35] The Maintenance RC viewer, version  5.0.7.328060 and dated August 9th was promotion as the de facto release viewer on Wednesday, August 23rd.  A new Maintenance viewer is expected to appear around the middle of week #35 (commencing Monday, August 28th).

[1:23] The Alex Ivy 64-bit viewer failed a QA test as a result of issues being introducing in refectoring how the SL Luncher code (used to determine whether the 32-bit or 64-bit version of the viewer should be installed on Windows systems). The issues aren’t serious, so it is hoped this viewer will appear some time in week #35.

[10:50] A further delay in promoting this version of the viewer, which currently remains at version 5.1.0.507412, is that accurate crash reporting data isn’t being gathers, due to issues with the back-end crash analyser, which are in the process of being addressed.  This problem isn’t uinque to the 64-bit build, but does seem to impact it the most.

[2:13] The Voice viewer, currently version 5.0.7.327253, dated June 21st, has been updated for parity with the release viewer, but has yet to pass QA testing at the time of writing.

This leaves the overall pipeline as follows:

  • Current Release version 5.0.7.328060, dated August 9, promoted August 23 – formerly the Maintenance RC
  • Release channel cohorts (please see my notes on manually installing RC viewer versions if you wish to install any release candidate(s) yourself):
  • Project viewers:
  • Obsolete platform viewer version 3.7.28.300847, dated May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

[2:50] It is likely the the Lab will, at some point in the future, move to block some of the older and more outdated versions of the official viewer from accessing Second Life.

Texture Memory Increase and Cache Improvements

[13:58] The 64-bit viewer build will see an increase in the texture memory size, although this isn’t part of the current Alex Ivy build. This work includes:

  • [14:23] Changes to texture caching, including some re-organising and optimisation, and improvements to  statistics gathering to assist in understanding how well it is performing.
  • [15:25] Changes to format of the stored texture data, using the post-decoding raw data, rather than the JPEG2000 data, which should “significantly” improve the amount of textures which can be loaded per second by the viewer, when visited a location previously cached.
  • [16:18] An expansion of the cache version guards so that if a TPV changes versions of their texture decoding software (e.g. KDU) or change from 32- to 64-bit, the cache will get wiped, and force the start of a new one (this is something Firestorm already does).
  • An increase in cache size.

Further changes will follow from these, including possibly changing how the static VFS cache is handled, but this work has yet to be fully characterised.

BUG-10515: Unable to Rez Due To Invalid Mesh Data

[24:42] The have been increased report of mesh rezzing failures with the message “Unable to rez object because its mesh date is invalid.” It has been noted across different objects, but multiple reports have been made for it occurring with the Maitreya Lara 4.1 body.

The problems seem to occur randomly on different simulators, and the Lab has had problems getting the issue to reproduce, as often a simulator restart may clear it, so Grumpity Linden is seeking reports of the problem which can be made as they occur (object, region name, date / time, etc.), so that an attempt to reproduce the error can be be made and log files immediately gathered before any restart takes place (and/or log files are lost).

Other Items

[3:32] Infrastructure work:  the events of Tuesday, August 22nd (see April Linden’s blog post) became an accidental way of testing some of the new infrastructure changes (in this case, the log-in servers) the Lab is implementing, and will continue to implement over the next several months. This encompasses Second Life and the Lab’s various web properties related to it. This includes further operating system updates as well as various component updates.

[24:09] Part of this work involves inventory, although whether it will lead to user-visible improvements to inventory handling is unclear at this point.

[4:19] Place Pages: this have received a number of fixes recently, for those who wish to use them (see here for general information on Place Pages if you are unfamiliar with them).

[5:32] Estate Tool ban list improvements: The lab did some initial work to improve ban lists at the estate / region level a while ago, but the intended work to improvement to overall layout for the lists, etc., has been delayed do to work on dealing with viewer crashes, etc. However, work will be resuming before the end of the year, and possibly sooner. However, this work is unlikely to see an increase in the number held by the ban list, but will focus on usability improvements.

[28:44]  McAfee Total Protection Issue: This is a problem being experienced by some users on TPVs (e.g. Alchemy and Firestorm) where the viewer is being flagged by McAfee Total Protection. This might be down to a code signing issue, however, to assist in further investigations, the problem needs to be reproduced and reported using the LL viewer.

[33:35] BUG-6925, HUDs and Attachments randomly detaching on region crossings / teleports: This has been investigated and determined to most likely be the result of a race condition, but a fixed has yet to be implemented.

[39:56-end] General discussion on documentation (e.g Axon animation; Objectllsd) and avatar physics calculations.

 

SL project updates week 32/3: TPV Developer Meeting

Whimberly, Whimberly; Inara Pey, August 2017, on Flickr Whimberly – blog post

The majority of the notes in this update are taken from the TPV Developer meeting held on Friday, August  11th 2017. The video of that meeting is embedded at the end of this update, my thanks as always to North for recording and providing it. Timestamps in the text below will open the video in a separate window at the relevant point for those wishing to listen to the discussions.

Server Deployments Week #32 – Recap

Please refer to the deployment notice for the week for latest updates and news.

  • On Tuesday, August 8th, the Main (SLS) channel was updated with a new server maintenance package (#17.07.27.327933), comprising “additional internal fixes”,
  • The scheduled deployment to the RC channels for Wednesday, August 9th has been cancelled due to a back-end data issue which meant the region channel names weren’t being set correctly, so they didn’t start.

SL Viewer

[00:54] The Maintenance RC viewer was updated on Wednesday, August 9th to version 5.0.7.328060. This is currently the most likely candidate for promotion to de facto release status. The rest of the LL viewer pipeline remains as at the start of the week:

  • Current Release version 5.0.6.326593, released on May 26, promoted June 20 – formerly the AssetHTTP RC viewer – overview – download and release notes
  • Release channel cohorts:
  • Project viewers:
  • Obsolete platform viewer version 3.7.28.300847, dated May 8th, 2015 – provided for users on Windows XP and OS X versions below 10.7.

[01:15-01:40 and 4:51-5:33] It is hoped the Alex Ivy 64-bit viewer will be updated in the early part on week #33 (commencing Monday, August 14th). Currently, the viewer is still showing an elevated crash rate for some, for reasons yet to be determined. In the meantime, the Lab is adding further code that will hopefully capture more crash information.

[01:56 and 06:49] The 360-degree snapshot viewer is currently on hold due to people being on vacation at the Lab. This may be a slightly extended delay while resources are diverted to the 64-bit viewer and CEF work.

[14:22-14:34] The Voice viewer is also suffering a high crash rate, also for unknown reasons.

[14:35-14:46] One of the next viewer projects for the Lab is a major re-vamping of the crash reporting and analysis system, which will hopefully make it easier to trace down probable causes for viewer crashes.

llHttpRequest Issues

[02:15-04:21] The least couple of weeks have seen server-side issues with llHttpRequest. Some of this has been due to low-level library changes that were inherited in updating the simulator code to some of the new components, including the OS itself. Other issues were the result of changes the Lab made at around the same time.

Most of these issues have now been rectified, although there is still some incompatibility issues still to be sorted. There is one further significant update related to this which will be going out on Aditi (the Beta grid), which will be put to the test with the intention of seeing if anything else breaks. A blog post / forum notice will precede this.

Simulator Infrastructure Updates

[08:52-09:28] The Lab is going to be carrying out a lot of simulator infrastructure work over the next two quarters. Efforts will be made to make user-visible updates during this work, but most of the work will be transparent to users, and may result in some performance improvements.

Other Items

[08:18-08:46] Group notice failures: No work has been put into looking at why some group notices are being dropped. This is currently seen as a lower priority than dealing with issues which can crash simulators.

[10:29-11:37] Dynamic mirrors (again): (see STORM-2055). The Lab has no plans to work on this at all, but Oz offered a *possible* compromise:

If an open-source developer implements mirrors that work, and are acceptable from a performance point of view, without unreasonable restrictions on where you can put them and what you can put them on, I at least will be happy to consider the contribution. But I don’t believe that’s possible, and I am assured by graphics developers who know a great deal more about rendering than I do, that it isn’t possible; and that’s why we’re not doing in. So, if you have a miracle worker out there who wants to do a contribution to the viewer – go for it! But since I’d like to get other work done, I’m not going to devote our developers to doing it.

[13:37-14:01] EEP – Environment Enhancement Project: a re-iteration of this project see the outline Google doc or my overview for more) will allow for things like phases of the Moon.

[18:20-18:32] Viewer UI Region / Estate ban list improvements: this work is still pending a developer resource.

[19:10-24:12] Abuse of Setting Home:  it has been reported that griefers are able to circumvent being kicked out of a region (TP Home) by moderators if they have home set to that region. The alternative to this is to ban them from the land – but this requires the rights to do so. There are also alleged means to bypass estate bans if the home position is set to the banned region – which should not be possible. If there is a bug allowing this to happen, it needs to be reported with repro steps.

PacMan Stars and Black Stars: the issue of “Pacman stars” (irregularly shaped stars) has been fixed via a contribution by Drake Aconis (Sovereign Engineer). However, the issue of stars appearing as black dots in some daylight windlight settings has yet to be fixed. This may be looked at as a part of the EEP.

 

SL project updates 30/3: TPV Developer meeting

Follow Your Bliss, Sea Foam; Inara Pey, July 2017, on Flickr Follow Your Bliss – blog post

The majority of the notes in this update are taken from the TPV Developer meeting held on Friday, July 28th 2017. The video of that meeting is embedded at the end of this update, my thanks as always to North for recording and providing it. Timestamps in the text below will open the video in a separate window at the relevant point for those wishing to listen to the discussions.

Server Deployments Week #30 – Recap

Please refer to the deployment notice for the week for latest updates and news.

Main (SLS) Channel

There was no deployment / re-start of the Main (SLS) channel on Tuesday, July 25th, the Main (SLS), which remains on server maintenance package #17.07.11.327548. This update included a fix to allow DJ boards to work, however, any scripts which have not been updated to meet the new requirements may not work. for details, please refer to this forum thread.

RC  Channels

The deployment to the RC channels did not go as planned, after it was discovered the release had a creeping crash bug that could not be reproduced on Aditi, but would be a bad experience for all regions on the RCs. A roll-back was therefore performed, leaving all three RCs on the same server maintenance package as the Main (SLS) channel (#17.07.11.327548).

SL Viewer

[0:10-3:33] The Maintenance RC viewer updated to version 5.0.7.327940 on Friday, July 28th. This update includes two crash fixes, and an update for a pull request fix, which Oz has asked third-party viewers to include as soon as possible. Essentially, some of the very low-level code in the log-in process has a very aggressive approach to retries if it encounters log in delays, and this needs rectifying.

This only became apparent when a main network cable in the Lab’s data centre was severed (by a backhoe!). This shifted log-in attempts to an alternative network connection where the combination of high user numbers and this aggressive retry loop in the code resulted in what was essentially a denial of service attack which overwhelmed the log-in servers. Given the depth of the code in the viewer, it is believed it is present in the vast majority (if not all) viewers. This also means the Maintenance viewer will get promoted relatively soon.

[3:59-5:16] The Alex Ivy 64-bit viewer still has an elevated crash rate and the Lab is making changes to obtain better data on the crashes. The 32-bit version of the viewer in particular is suffering “really high” crash rates.

The remaining viewers in the pipelines remain unchanged:

Web Property, Attachments and ToS Issues

[5:48-7:01] Some users have been experiencing some or all of the following issues during the week:

  • Inability to access various LL web properties (e.g. their own dashboard, the JIRA system, etc). As of the TPV Developer meeting, all web services were believed to be back up and running, although the Lab weren’t sure as a to the cause of the issues.
  • Attachment failures – either on attach or with scripts. These are still being investigated, and a definitive cause has not yet been found.
  • The Terms of Service update, which officially comes into force on Monday, July 31st but which currently requires acceptance by users, has been causing log-in issues for some. Again, the cause isn’t as yet known.

Other Items

Parcel Sounds

[7:55-13:12] There is a discussion on parcel sounds (from objects and avatars). In short: parcel sounds can be restricted from within a parcel going out, but not for sounds from other parcels coming in. As parcel privacy is two-way, this has caused some confusion for some users. The Lab is hesitant to change the scope of the setting (e.g. make it apply to both sounds leaving and entering a parcel), they are possibly amenable to changing the setting to sounds are restricted to parcels by default, or receiving a code contribution / feature request to separately stop sounds entering a parcel.

Sansar Public Beta

[17:58-18:50] During the meeting, Oz Linden indicated that the Sansar public Creator Beta will be launching on Monday, July 31st / during the week commencing Monday, July 31st.

However, during the week of July 24th through 28th, the Lab hosted a series of meetings involving staff, those of us fortunate enough to be in Sansar and creators and bloggers from Second Life who have not been a part of the Creator Preview in which it was indicated the company wasn’t ready to provide a date for the public Beta, other than it being “soon”.

We’ll obviously find out what is happening on Monday, but either way, I’ll have at least one article on Sansar and Second Life coming out quite soon which will delve into the meat of the meetings – so to speak!

Lab’s Second Life Development Summit

[18:00-18:42] Week 31 (commencing Monday, July 31st) marks the next Second Life summit in which Lab staff directly involved in running and maintaining SL (developer, product team, operations, etc), get together to plan out the next several months of work and projects for the platform.

In Brief

[19:51-20:36]  Further Estate Tool Improvements: the work to improve the Region / Estate floater in the viewer to make it easier to manage ban lists, etc., is still in-hand at the Lab, it’s just been delayed due to other work being carried out / completed.

[21:26-22:57] Premium “Goodies”: Grumpity Linden indicated there are some “awesome goodies” in the pipeline for Premium members, which sparked a lot of text-based suggestions.

[22:57-25:46] Group notices being dropped: there are issues around group notices being received by users in certain groups. The Lab has made adjustments to handling the UDP packets which handle group notices to try to improve things, but the issue of dropped notices is hard to reproduce, and therefore hard to diagnose and fix.

[26:51] Place Pages: have been updated to include events.

 

SL project updates 27/3: TPV Developer meeting and JIRA

Stony Hill Vineyard, Stony Hill; Inara Pey, June 2017, on Flickr Stony Hill Vineyard – blog post

Friday, July 7th saw a truncated TPV Developer meeting take place in lieu of the planned meeting for Friday, June 30th. As always, the video of the meeting is embedded at the end of this report, and my thanks to North for recording and providing it. Timestamps in the text below refer to that video, and will run the video from that point in a separate browser tab.

SL Viewers

[3:30] The Alex Ivy 64-bit release candidate viewer (version 5.1.0.507006 at the time of writing) is due to be updated in week #28 (commencing Monday, July 10th). The viewer is liable to receive further updates prior to any promotion to release status, as the Lab further refines the Windows SL Launcher.

As previously noted in these pages, the launcher is designed to detect which version of the viewer a Windows system can run. If it detects the system can run the 64-bit version of the viewer, it downloads and installs that, if not, it defaults to downloading the 32-bit version. An upcoming update to the viewer will include a debug setting to override this and force one or other version to be downloaded and installed (obviously, this will not work trying to run the 64-bit viewer on 32-bit Windows, but will allow the 32 viewer to be run on 64-bit Windows).

Crash rates for the viewer are elevated, but the Lab suspects this is down to the number of 32-bit versions being run.

[5:35] This leaves the current viewer pipelines as:

  • Current Release version 5.0.6.326593, released on May 26th, promoted June 20th – formerly the AssetHTTP RC viewer – overview
  • Release channel cohorts:
  • Project viewers:
  • Obsolete platform viewer version 3.7.28.300847 dated May 8th, 2015 – provided for users on Windows XP and OS X versions below 10.7.

The 360 snapshot viewer should be updated with some fixes “soon”.

Server Updates

[6:05] The simulator version using the new Linux operating system update, currently deployed to the Magnum RC channel (server version 17#17.06.29.327400) should be promoted to the Main (SLS) channel on Tuesday, July 11th.

[12:07] This simulator version also includes the fix for DJ boards (see BUG-10073). Some people may need to update their board scripts.

JIRA Temporarily Closed to Comments

The Second Life JIRA has been temporarily closed to comments, because: idiots. Or to allow the Lab to explain it:

Recently, our bug reporting system (Jira) was hit with some spam reports and inappropriate comments, including offensive language and attempts at impersonating Lindens. The Jira system can email bug reporters when new comments are added to their reports, and so unfortunately the inappropriate comments also ended up in some Residents’ inboxes.

We have cleaned up these messages, and continue to investigate ways to prevent this kind of spam in the future. We appreciate your understanding as we work to manage an open forum and mitigate incidents like this.

In the short-term, we have disabled some commenting features to prevent this from recurring. This means that you will not be able to comment on Jiras created by other Residents. We apologize for this inconvenience as we look into long-term solutions to help prevent this type of event from occurring.

Other Items

  • The next Firestorm release is liable to be in September
  • [8:51] The Lab is looking at turning off the remaining UDP messaging for asset delivery on the servers in February 2018, although this is TBC. This means that viewers not running the latest HTTP updates will not be able to receive any asset data.
  • [17:41] The next scheduled TPV Developer meeting is Friday, July 28th