2019 SL User Groups 19/2: Content Creation summary

Ukivok; Inara Pey, March 2019, on FlickrUkivokblog post

The following notes are taken from the Content Creation User Group (CCUG) meeting, held on Thursday, May 9th 2019 at 13:00 SLT. These meetings are chaired by Vir Linden, and agenda notes, meeting SLurl, etc, are usually available on the Content Creation User Group wiki page.

Environment Enhancement Project

Project Summary

A set of environmental enhancements allowing the environment (sky, sun, moon, clouds, water settings) to be set region or parcel level, with support for up to 7 days per cycle and sky environments set by altitude. It uses a new set of inventory assets (Sky, Water, Day),  and includes the ability to use custom Sun, Moon and cloud textures. The assets can be stored in inventory and traded through the Marketplace / exchanged with others, and can additionally be used in experiences.

Due to performance issues, the initial implementation of EEP will not include certain atmospherics such as crepuscular rays (“God rays”).

Resources

Current Status

  • The next viewer RC update is “close”. This should:
    • Include a batch of shader updates that will hopefully improve things “considerably”.
    • Be merged to the current LL release viewer (version 6.2.1.526845 at the time of writing, promoted May 7th).
  • An issue of late has been that of new bugs popping up as existing ones are dealt with.
  • A recently noticed issue has been with day cycles – cloud scrolling across the sky will suddenly “hiccup” slightly in their motion, rather than scrolling smoothly throughout.

Animesh Follow-On

  • Vir continues to work on adding shape support (or similar) to Animesh, specifically on the infrastructure requirements for being able to send slider parameters for Animesh objects to and from the viewer.
  • No decision has as yet been made to whether to offer full body shape support or to use individual parameters, but the underpinning infrastructure requirements for both are more-or-less the same.
  • The focus is therefore on developing the infrastructure (messaging, etc), to a point where both can be tested as a decision made on the basis of that testing.

Bakes On Mesh

Project Summary

Extending the current avatar baking service to allow wearable textures (skins, tattoos, clothing) to be applied directly to mesh bodies as well as system avatars. This involves viewer and server-side changes, including updating the baking service to support 1024×1024 textures, but does not include normal or specular map support, as these are not part of the existing Bake Service, nor are they recognised as system wearables. Adding materials support may be considered in the future.

Resources

Current Status

  • There is a simulator change pending. This includes a means of accessing BOM UUIDs.
    • These were changed in the last back-end update as a result of underlying asset property issues. If there is BOM content using the old UUIDs, this will have to be updated.
    • The simulator update is intended to allow access to the texture UUIDs without having to do so numerically, as is currently the case. This should re-enable the ability to access them via their name abbreviations.
  • There is also a further Appearance Service change pending, deigned to correctly handle tattoo layer with partial transparency (currently, if a tattoo with partial transparency is sent for baking via the new BOM channels without any underlying opaque layer, then the alphas are not correctly resolved). This is with the Lab’s QA team.
  • There are no BOM-specific updates pending to the RC viewer, but this will, as per the Lab’s viewer release policy, be updated to maintain parity with the latest release version of the viewer (6.2.1.526845) at the time of writing.
  • Given the overall status of BOM and EEP, it would appear likely that BOM could be promoted ahead of EEP as a released project and viewer.

2019 SL User Groups 19/1: Simulator User Group

A L T I T U D E; Inara Pey, March 2019, on FlickrA L T I T U D Eblog post

Server Deployments

As always, please refer to the server deployment thread for the latest news.

  • On Tuesday, May 7th, the SLS (Main) channel updated to server maintenance package 19#19.04.25.526669, primarily intended to correct the simulator-side EEP regressions that resulted of the roll-back of Thursday, April 18th. This update also includes a number of other internal fixes.
  • On Wednesday, May 8th, the three RC channels should be updated with server maintenance package #19.05.01.526856, comprising internal fixes.

SL Viewer

The Teranino Maintenance RC viewer updated on Friday May 3rd, to version 6.2.1.526845. This viewer was subsequently promoted to de facto release status on Tuesday, May 7th.

The remaining viewer pipelines remain unchanged:

  • Release channel cohorts:
    • EEP RC viewer version 6.2.0.526104, dated April 11th.
    • Bakes on Mesh RC viewer, version 6.1.1.525409, dated March 26th.
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17th, 2017 and promoted to release status 29th November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, dated May 8th, 2015 – provided for users on Windows XP and OS X versions below 10.7. This viewer will remain available for as long as reasonable, but will not be updated with new features or bug fixes.

Teleport Disconnects

Following the deployments of server maintenance package 19#19.04.25.526669, it appears that there has been a significant improvement in teleports, with Simon Linden noting that the “TP crash has been mitigated and success rates are back to what they were months ago.”

This doesn’t mean that the Lab is ceasing efforts on trying to improve teleports and region crossings; rather that the priority of the work over that last few weeks has been on easing the problems people have been experiencing the last month.

Script Processing Issues

As reported in my week #18 SUG summary, there have been numerous reports of script run time issues, with some reporting that problems only started occurring following the roll-back on April 18th, 2019. BUG-226851 outlines some of the problems.

There is no specific update on this, other than the Lab’s engineering team has been asked to put together ways for them to look at script run times and performance grid wide.

2019 SL User Groups 18/1: Simulator User Group

Toshi Farms; Inara Pey, March 2019, on FlickrToshi Farmsblog post

Server Deployments

As always, please refer to the server deployment thread for the latest news.

  • There was no deployment of the SLS (Main) channel on Tuesday, April 20th, leaving all regions on that channel on server update 19#19.04.22.526534 comprising performance improvements (possible fixes for teleport / region crossing issues) and additional internal logging.
  • On Wednesday, May 1st, all three main RC channels should be updated to server maintenance package 19#19.04.25.526669, primarily intended to correct the simulator-side EEP regressions that resulted of the roll-back of Thursday, April 18th. This update also includes a number of other internal fixes.

SL Viewer

There have been no viewer updates at the time of writing this update, leaving the viewer pipelines as follows:

  • Current Release version 6.2.0.526190, formerly the Estate Access Management RC viewer, dated April 12, promoted April 17 NEW. – see my EAM overview for more information
  • Release channel cohorts:
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7. This viewer will remain available for as long as reasonable, but will not be updated with new features or bug fixes.

My.secondlife.com

My second Life has been down since the weekend of April 27/28th, 2019. This has affected all access to the site, the upload of snapshots and in-world web-based profiles.

Linden Lab is aware of the issue, and working to resolve the outage.

Teleport Disconnects

The teleport disconnect issue is now being seen as a race condition,with Simon Linden commenting:

The TP issue looks like a race condition … we “fixed” 3 other bugs and made that worse.

To which Oz Linden added:

And we’ve already put in place a bunch of improvements to our RC monitors to help prevent a similar recurrence (with more coming soon) … Doubtless Murphy will find new ways to avoid our monitors from time to time, but one tries.

The Lab now feel they have investigated the issue and gathered sufficient data around it to be able to perform a post-mortem on the situation to (hopefully) reduce the likelihood of such an occurrence in the future.

Script Processing Issues

This was raised at the April 26th TPVD meeting as well. There have been numerous reports of script run time issues, with some reporting that problems only started occurring following the roll-back on April 18th, 2019. BUG-226851 outlines some of the problems.

As per the TPVD meeting, the Lab are not aware of anything that may have changed to impact script run time (particularly on Full regions); but the problem has been somewhat exacerbated by the issue being bounced between the JIRA and support (note how the above report has been closed, referencing the matter back to support).

2019 SL User Groups 17/3: TPV Developer Meeting

Authors Point; Inara Pey, March 2019, on FlickrAuthors Pointblog post

The following notes are taken from the TPV Developer meeting held on Friday, April 26th, 2019. A video of the meeting is embedded below, my thanks as always to North for recording and providing it. This was a relatively short meeting, with several periods of audio silence and text chat. The key points of discussion are provided below with time stamps to the relevant points in the video, which will open in a separate tab when clicked.

SL Viewer

[00:00-01:17] There have been no SL viewer updates this week, leaving the pipelines as follows:

  • Current Release version 6.2.0.526190, formerly the Estate Access Management RC viewer, dated April 12, promoted April 17 NEW. – see my EAM overview for more information
  • Release channel cohorts:
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7.

The EAM viewer is awaiting one simulator side update; the next RC viewer likely to be promoted to de facto release viewer will probably be the Teranino Maintenance RC viewer.

Teleport (and Region Crossing) Issues

[1:20-3:35]

Disconnects

  • Major effort has been put into trying to resolve the teleport disconnect issue – hence the multiple deployments this week.
  • In the process of developing and deploying the fixes, the Lab has significantly improved its ability to monitor teleports.
  • This improved monitoring / stats gathering will help baseline teleports for future reference, should further issues crop up.
    • It appears to suggest teleport success rates are now significantly better than prior to the updates being deployed.
    • The added monitoring has no appreciable impact on performance.

Attachment Loss on Teleport

  • Progress has continued with fixes for the issue of attachments coming off / becoming ghosted/ etc., as a result of a teleport / region crossing.
  • It will likely be a couple of weeks before these are deployed server-side, as the Lab will be keeping an eye on the teleport / region crossing disconnect issue to ensure the fixes that have been deployed really do help break down the issues that have been experienced.

Snapshots: Flickr Cap Fail and Withdrawal of Facebook Support

[4:15-5:50]

Flickr Cap Fail

There is an issue with the snapshots to Flickr capability failing see BUG-226826). This appears to have perhaps been an unannounced change at the Flickr / Smugmug end of things, rather than anything the Lab has done, but investigations are still ongoing.

Update April 30th: the Flickr cap fail now appears to be fixed. 

Facebook Support

It was announced on Friday, April 19th, that support for uploading snapshots from the viewer to Facebook has now been completely  withdrawn. As such, the viewer-side code is to be shortly removed, with the Lab noting:

Due to continued changes in the Facebook API, as of today the Second Life viewer will no longer be able to support Facebook Connect for sharing your in-world photos and posts.  We apologise for this inconvenience and will be removing the UI from the viewer shortly. We will, of course, be happy to see your SL posts on Facebook going forward, and you can always say hello and check out what’s happening on our official page: https://www.facebook.com/secondlife.

Script Processing Issues

[6:50-13:45]

Issues with script processing have been raised at a number of meetings recently, and were mentioned again at this TPVD.

  • Specifically, it was reported at this meeting that since the April 18th roll-back / update, some Full regions seem to have script run-time capped 12ms, so only around 60% of scripts are run, while some homesteads appear to only run around 20% of scripts..
  • As it was also reported that these affected regions were running OK following the initial deployment of the newer version of the OS, this issue appears to be specifically related to the changes made on April 18th.
  • The Lab is not aware of anything having been changed that might have impacted script run-time.
  • The newer version of the operating system (which is specifically required as a part of preparing the simulators to a cloud-based infrastructure) is due to be further deployed to simulators Agni (the main grid) in week #18. This should provide the Lab with a broader cross-section of simulator running both the older and newer versions of the OS, which will allow a more informed comparison of metrics from the two versions to be made, which could help determine if there is a more broad-based issue with scripts on the newer OS version.

Group Notices to IM

[16:00-22:33]

  • The Lab is considering the possibility of no longer sending group notices to e-mail when a user is off-line.
  • Right now, if IM to e-mail is enabled for when a user is off-line from SL, it will allow both IMs and group notices to be forwarded to e-mail. This can result in message volumes / content being seen as spam.
    • As it is invariably the secondlife.com domain that is linked to such reports, this can lead to it being regarded as a spam site, degrading the ability for secondlife.com to deliver e-mails in general, as e-mail services mark the domain as an originator of spam.
    • This move is therefore intended to make e-mails from the secondlife.com domain for reliable.
    • Although other forms of messaging forwarded to e-mail can potentially add to the problem of spam labelling (e.g. object IMs to e-mail), group notices are seen as by far the biggest cause.
  • No decision on this has been taken either way, but disabling the ability for group notices to e-mail is easier than other options (such as requesting user go through all their groups and disable group notices).
  • Were this change to be implemented, then only IMs would be sent to e-mail; group notices received when off-line would be held until the user next logs-in, as is currently the case.
    • It has been suggested that increasing the limit on the number of messages that can be queued when a user is off-line might ease the blow of blocking group notices being sent to e-mail. While this idea is already under consideration, no decision has been taken either way.

[25:06-26:18] If this idea does move forward, it is hoped that changes being planned to the way SL events work will reduce the need for at least some group messages.

  • No specifics were available on the updates that are being planned for events, but it has already been suggested that the updates include an API to allow events to be properly listed on viewer log-in screens (see feature request BUG-226867).

In Brief

  • [13:58-15:00] Following the release of a video on You Tube by a Lab employee, the question was asked about the Lab’s policy on staff using TPVs. In short:
    • Lab staff must use the official viewer on Lab equipment and/or their official Lab accounts.
    • Lab staff may use any viewer of their own choosing on their own hardware and when using their personal user account.
      • This was actually the case with the video in question: it was filmed and released via a personal account, not an employee account.
  • [31:43 (text)-35:30] Questions continue concerning LL’s support of 32-bit operating systems (it was mentioned that Microsoft will apparently stop supporting 32-bit “with the next [Windows 10?] update”). However, one of the reasons for LL’s continued support of 32-bit Windows is the number of users on less capable / 32-bt specific GPUs.
  • [35:40-36:25] At least some of the EEP regressions witnessed following the April 18th roll-backs / redeployments should hopefully be corrected with the simulator updates due in week #18.
  • [39:36-40:56] It’s often asked when simulator updates aren’t “more thoroughly tested” by the Lab. Simply put, such is the number of Agni (main) grid simulators coupled with the wide variety of ways people use Second Life (think of all the different in-world scripted object, or even all the purpose-built, custom windlights, for example), that replicating it all in a comprehensive test environment simply isn’t possible. Hence why the Lab use the release candidate channels: while testing is carried out (and the Lab is constantly trying to improve its test environments), the RCs provide a further means of “testing the water” before deploying updates grid-wide.
  • [42:39-43:11] video playback support (MP4): work is on the roadmap for this, but the Lab has yet to get to it.

2019 SL User Groups 17/2: Content Creation summary

Candlewood; Inara Pey, March 2019, on FlickrCandlewoodblog post

The following notes are taken from the Content Creation User Group (CCUG) meeting, held on Thursday, April 25th 2019 at 13:00 SLT. These meetings are chaired by Vir Linden, and agenda notes, meeting SLurl, etc, are usually available on the Content Creation User Group wiki page.

There was little to report, project-wise, this meeting, with most of the time taken up with a general discussion on avatar complexity, LI, ARCTan, LODs and efficient content. This summary focuses on the project updates that were offered.

Environment Enhancement Project

Project Summary

A set of environmental enhancements allowing the environment (sky, sun, moon, clouds, water settings) to be set region or parcel level, with support for up to 7 days per cycle and sky environments set by altitude. It uses a new set of inventory assets (Sky, Water, Day),  and includes the ability to use custom Sun, Moon and cloud textures. The assets can be stored in inventory and traded through the Marketplace / exchanged with others, and can additionally be used in experiences.

Due to performance issues, the initial implementation of EEP will not include certain atmospherics such as crepuscular rays (“God rays”).

Resources

Current Status

  • The regressions in environment appearance that have been seen since the Thursday, April 18th roll-backs will hopefully be corrected with the next simulator deployment that has EEP included.
  • Shader issues are continuing to be investigated and resolved as they come in / can be fixed.
    • The next release candidate version of the viewer may address the problems of environments looking unnaturally dark in the EEP viewer.
    • Graham Linden is looking at some updates provided by user Geenz Spad, who initially formulated how materials could be added to Second Life.
  • BUG-226752 “[EEP] Interest Lists Culling – Draw Distance has little effect on scene rendering” – still has yet to be addressed.

Animesh Follow-On

  • Vir continues to work on adding shape support (or similar) to Animesh, specifically on the infrastructure requirements for being able to send slider parameters for Animesh objects to and from the viewer.
  • Some of this involves using the infrastructure developed for EEP.
  • This work is still in its early days.

Bakes On Mesh

Project Summary

Extending the current avatar baking service to allow wearable textures (skins, tattoos, clothing) to be applied directly to mesh bodies as well as system avatars. This involves viewer and server-side changes, including updating the baking service to support 1024×1024 textures, but does not include normal or specular map support, as these are not part of the existing Bake Service, nor are they recognised as system wearables. Adding materials support may be considered in the future.

Resources

Current Status

Anchor Linden continues to work with some appearance service issues that need to be fixed before the project can progress.

Starter Avatars

As I noted at the time, several of the last batch of Starter Avatars for Second Life, released in January 2019, came sans any form of animation override (see: More Classic starter avatars for Second Life). This has now been addressed, and those avatars that needed them have now been updated with AOs.

Avatar Complexity / Impact – Summary

The subject of avatar complexity and lack of any Land Impact control for avatars formed a major point of general discussion in the meeting. Obviously, there is ARC – and the upcoming ARCTan project that is re-evaluation a range of rendering costs, including in-world objects and avatars. However, unlike LI, avatar rendering costs itself isn’t really an incentive to build efficient worn content for avatars.

However, trying to be more proactive with avatar complexity is difficult. Take the idea of some form of “Avatar Impact” akin to Land Impact:

  • How should it be defined?
  • How should individual attachments be weighted? purely on their in-world LI? Number of vertices? Tri / poly count? A combination?
  • What sort of policy needs to be put in place?
  • What happens if an avatar tries to enter a region / parcel where its “AI” exceeds the land capacity? Should a simplified version of the avatar be allowed?
  • How will a home owner feel if they find they cannot rez a new item of furniture because it would exceed their land capacity because their “AI” is too high?

As such, any such “avatar impact” would require a substantial changes to Second Life – as well as a lot of lead-time and explanation to users. So while not impossible, implementation would have to be weighed carefully. Currently, there are no plans to introduce any such system – and the target remains on being able to move forward with ARCTan.

This led to a broader discussion on complexity, the potential of ARCTan and a slight segue into LOD and auto LOD (again, something that might had some advantages – and disadvantages – but really not now suited to SL).

2019 SL User Groups 17/1: SUG – teleport disconnects update

Puddlechurch; Inara Pey, March 2019, on FlickrUmiblog post

Server Deployments

From the server deployment thread for the week:

We are working on the TP & sim crossing disconnect issue, and making several changes over this week.  These may be a little bit more disruptive than our usual grid roll process and we apologise in advance for the inconvenience of those.  We’ll do our best to keep this disruption to a minimum. We have a simulator update which we will roll to BlueSteel and LeTigre on Tuesday.  Depending on the results we will make additional plans for gridwide rolls as soon as practical. We thank you for your patience and fully realize both the urgency and the frustration this has been causing.

Thus far, there has been a deployment to BlueSteel and LeTigre – server update 19.04.22.526534. Check the deployment thread for further updates.

SL Viewer

There have been no viewer updates at the start of the week, leaving the viewer pipelines as follows:

  • Current Release version 6.2.0.526190, formerly the Estate Access Management RC viewer, dated April 12, promoted April 17 NEW. – see my EAM overview for more information
  • Release channel cohorts:
  • Project viewers:
  • Linux Spur viewer, version 5.0.9.329906, dated November 17, 2017 and promoted to release status 29 November – offered pending a Linux version of the Alex Ivy viewer code.
  • Obsolete platform viewer, version 3.7.28.300847, May 8, 2015 – provided for users on Windows XP and OS X versions below 10.7. This viewer will remain available for as long as reasonable, but will not be updated with new features or bug fixes.

Teleport Disconnects

Let’s see … for server news, many of us are focused on the teleport issues … I hate to promise anything, but we all know a lot more about TP internal problems than we used to 🙂 . We do understand they’re annoying and a frustrating problem – we definitely want to fix it and make it better.

– Simon Linden, Simulator User Group, April 23rd

From the above statement, it should be clear that the Lab is devoting a lot of time to teleport disconnects. There have been mixed reports on the outcome of the deployment made on Thursday, April 18th, 2019, with some indicating matters have improved, others reporting no real change. In particular, people visiting Fantasy Faire and travelling around the new Linden Homes continent appear to be faring a lot better than had been the case. Which is not to say the issue has in anyway been resolved – hence the continuing work.

While it had been indicated that the recent operating system update may have played a role in the problems, Simon again referenced the timing issue with region crossings, whilst also mentioning the potential for the simulator side of EEP possibly also playing a role in things.

One of the puzzles we’re trying to sort out is if somehow the environmental work caused the TP problems – the timing of the release is suspicious but the functionality _should_ be different …. The disconnects are a problem where the viewer and 2nd region don’t start talking as they should [so something is out of sync] or there’s a failure to communicate. I know, for example, the 2nd region is waiting for the viewer to connect and get a message … that never happens …

He continued:

It doesn’t seem to be closely associated with AV complexity … that said, the more complex your AV is, the more work it needs to change regions. It’s always been better to have fewer scripts and data for teleports and region crossings.

The problem is still trying to pin down actual potential causes, with disconnects remaining inconsistent in terms of reproduction., again as Simon noted:

It’s frustratingly inconsistent. It’s a lot easier to fix something that breaks all the time … to fix it, and to know when you fixed it. For example, before this meeting we ran a test that had probably 400 or so successful teleports, no disconnects … that’s good, but not proof of a fix.

Mazidox Linden, from the Lab’s QA team further qualified the Lab’s problem:

We need in the neighbourhood of 10000 teleports to have any kind of real statistical confidence, just as a reference point 🙂 .

EEP / Windlight Issues

Following the Thursday deployment, many region holders / designers noted significant differences in how their region windlights were being rendered. The didn’t get much in the way of discussion during the meeting, however, Rider Linden offered an apology for the situation, noting that the focus on the TP issue(s) resulted in some unexpected regressions. However, it’s not currently clear what might be done to deal with this issue.