On Tuesday, May 19th the Main (SLS) channel received the server maintenance package previously deployed to the three RC channel, comprising Internal server logging changes, back-end system bug fixes and a change to Reply-To e-mail addressing on snapshots. There were no RC deployments on Wednesday, May 20th.
SL Viewer
The Attachments Viewer RC (Project Big Bird) was updated to version 3.7.29.301943 on Thursday May 21st. As noted in part 1 of this week’s report, the initial RC release of this viewer had an elevated crash rate compared to the current release viewer, including a crash-on-exit bug, so this release will hopefully address those issues.
Group Chat
A fix for issues around BUG-9130, where some people were unable to see any posts in some or all of there group chats, including their own posts, while everyone else in the same group could see their posts, has started to be deployed across the chat servers, and should be completed on Friday, May 22nd.
“The chat servers got stuck with bad info about where the sender was, so the messages never reached them,” Simon Linden said at the Server Beta User Group meeting on Thursday, May 21st, reiterating an explanation given at a recent Simulator UG meeting. “And unfortunately it wouldn’t fix with relogging or even a chat server restart.”
“Loading…” Issue with Names in Group Chats
This is a viewer-side problem which causes avatar names to appear as “Loading” under certain circumstances in group chat (see BUG-3829 and STORM-2114). A contribution by Ansariel Hiller is currently with the Lab and is expected to be released as a part of the next Snowstorm contributions viewer, which is expected to appear soon.
Other Items
Region Restart Glitch
There has been something of a rise in reports of regions experiencing issues following recent following restarts – most noticeably caps failures. This is something the Lab is looking into, and Simon commented, “we have a suspicion that after rolls, as that server host starts up regions, it’s doing enough of them at about the same time that things get overloaded. It’s still a theory but makes some sense why we’d get cap failures like that.”
The following notes are primarily taken from the TPV Developer meeting held on Friday, March 27th, a video of which is included towards the end of the article (my thanks as always to North for recording it and providing it for embedding), and from the Server Beta meeting held on Thursday, March 26th. Any time stamps contained within the following text refer to the TPV developer meeting video.
Server Deployments Week 13 – Recap
As always, please refer to the deployment thread in the forums for the latest updates / news.
On Tuesday, March 24th, the Main (SLS) channel received the server maintenance package deployed to the three RCs in week 12, comprising updates which allow the Lab to make various configuration changes without having to necessarily run a rolling restart when they have done so. It contains not actual functional changes to the simulator software
On Wednesday, March 25th, the three RC channels received the same new server maintenance package, which is focused on inventory loss issues, and provides the Lab with better error detection and logging, improving their ability to look at some of the failure places and the removal of unused code. This updates does notremove the server-side messaging used in support of RTLP.
SL Viewer Update
Avatar Layers Project Viewer
Vir Linden’s work on a new global limit for system layer clothing was released as a project viewer, version 3.7.26.299805. With this viewer, a user can wear any combination of clothing layers (wearables), up to a maximum of 60, rather than being limited to (in general, and as with the official viewer) to a maximum of 5 items per layer type. Note that these changes do not apply to body part wearables (skin, shape, hair, eyes), for which the limit is still one of each, and do not affect attachments, for which the limit is still 38 total.
[07:18] There is already an update in the pipe for this viewer, which should be appearing next week.
Camera Positioning / Handling
[05:12] While there are no specific details as yet, the lab is hoping to put some work into improving camera positioning and handling in the not too distant future, in the hope of removing various glitches and issues.
Build Tools Viewer
[05:54] There have been a few fixes added to this viewer (currently version 3.7.26.299443), so a further update to the release candidate version is with the Lab’s QA team and should be appearing in week #14 (week commencing Monday, March 30th).
Maintenance Release Viewer
[06:29] Currently at version 3.7.26.299845, the latest Maintenance release viewer has a range of issues, many of which have hopefully been addressed with a series of fixes, so an update to that viewer is also with the Lab’s QA team. However, given the scope of the updates, it is proving a little harder to pass the QA process.
Experience Tools Viewer
[06:50] The Experience Keys / Tools viewer (currently version 3.8.0.299338) is being merged-up with the latest release version of the viewer code (version 3.7.26.299635). The updated version should also be appearing (again as an RC) in week #14.
Viewer Code
[17:27 – 19:50] There is an interesting discussion on the viewer code which, for anyone interested in how the viewer has developed over the years, and how much of it dates back some 14 years.
Viewer-Managed Marketplace
[00:00] There was a pile-on test of the new Viewer-Managed Marketplace capability on Aditi in week #12, and Brooke Linden was at the TPV Developer meeting to provide feedback. The pile-on test did not reveal any significant issues in terms of performance.
However, there is still a viewer / simulator / marketplace communications issue which has to be resolved, which may take another couple of weeks to fix. After that, there are two grid deployments which need to take place: one for the VMM code itself, and one for updates to the Advanced Inventory System (AIS), so it is unlikely VMM will be fully deployed within the next month to two month, and the project viewer (currently version 3.7.25.298865) is unlikely to progress through a release candidate to release status until after the server components have been deployed.
Group Chat
Simon Linden has been working on significant improvements to the group chat service
[07:32] Simon Linden has been continuing to work on the group chat code, and all of his current updates should have been deployed to the back-end group chat servers. A broad consensus is that the issues which recently occurred as a result of some changes have been reversed, and that the group chat service as a whole is now running a lot better, both in terms of the early performance improvements Simon made, and with regards to the overall stability of the service and the servers.
[08:24] There is a further round of updating in the planning, but these require a platform upgrade to be carried out for the group chat service first. Therefore, unless unless the latest set of updates deployed by the Lab start to show issues, the engineering team will be switching focus for the immediate future, and will return to working on group chat once the necessary upgrade work has been completed.
Experience Keys / Tools
[09:20] One of the items the engineering team want to focus on in particular is Experiences, and getting the remaining back-end issues sorted out so that Experiences can be properly deployed.
Voice Updates
[09:59] There will be a further round of voice updates which are expected to appear in a project viewer “shortly”. They include (but are not limited to) things like general code clean-up to prevent unnecessary list loading, removal of media messaging in person-to-person calls (which has never worked), fixes for issues related to microphone volume and improvements to the microphone test so that you can now hear yourself when testing your microphone, and improvements for hot swapping microphones / headsets.
[13:58] There is some confusion over whether or not a fix to voice designed to prevent someone’s voice channel being “left behind” when teleporting between regions has actually worked. It had been thought that the fix for this had been deployed in later 2014. However, bug reports are still being made still reporting issues (see BUG-8543 and STORM-2109), prompting the Lab to re-examine the status of the fix.
[19:54] Voice package updates from Vivox are also expected to be forthcoming in the future as well.
Restore To Last Position (RTLP)
Oz Linden – keeping an eye on feedback through the Firestorm blog on “restore to last position”
[21:08] There have been around 400 responses to the Firestorm call for feedback on how people use the Restore To Last Position functionality found in some TPVs. As I’ve previously reported, the Lab had been considering deprecating the server-side message RTLP uses as an overall part of on-going work to reduce the amount of inventory loss issues (real or perceived) which can occur.
Firestorm’s call is helping the Lab to better understand how, as faulty as it might be, RTLP does fulfil a range of useful / valid use cases. Commenting on the fact the he has been reading through the feedback, Oz Linden said:
[21:49] Well, I understand that there are user scenarios that need to be addressed and need to be better supported. Whether the existing feature is the way to do that or not, I still consider to be an open question. I do want to take those use cases and work back through that process [of determining how best to serve them].
So the Lab still isn’t going to do anything “quickly” either way on RTLP, and people needn’t worry about RTLP vanishing / breaking “suddenly”.
In the meantime, they are working on other changes intended to address various rezzing failure situations. This work is more server-side focused, although it may be a while before updates appear on the grid as the exact nature of the updates is still being determined.
[23:42] Oz also again thanked everyone who responded to the Lab’s call for feedback on inventory losses in general, defining the feedback as “really, really useful”.
The following notes are primarily taken from the TPV Developer meeting held on Friday, March 13th, a video of which is included at the end of the article (my thanks as always to North for recording it and providing it for embedding), and any time stamps contained within the following text refer to it.
Server Deployments Week 11 – Recap
As always, please refer to the sever deployment thread for the latest updates and information.
There was no Main (SLS) channel deployment on Tuesday, March 10th
On Wednesday, March 11th, all three RC channels received the same new server maintenance package comprising “internal improvements for premium users”.
SL Viewer
The Avatar Hover Height viewer reached the release channel on March 10th, with the release of an RC version (3.7.26.299635). Avatar Hover Height allows you to adjust the vertical position of your avatar within some preset limits. See the wiki page and my overview.
This brings the total number of RC viewers in the viewer release channel to four, however:
[0:41] It is unlikely the Maintenance RC viewer (currently version 3.7.26.299610, released on March 6th) will be promoted without further update, as it has been found to contain a significant number of additional bugs which require fixing
[0:51] As the Avatar Hover Height RC viewer has only just been released, it is unlikely that the Lab will have enough stats on it to judge whether or not it can be promoted to the de facto release viewer in the immediate future; it is therefore likely to remain at RC status for at least another week, although initial reports suggest it is stable and doesn’t hide any unpleasantness
[01:07] The back-end support for Experience Keys / Tools still isn’t ready for the service to go live, although the Lab is making further progress with whatever needed to be done; it is therefore remains unlikely the that Experience Keys viewer (currently version 3.8.0.299338, released on March 9th) will be promoted to the de facto release viewer until such time as the remaining back-end work has been completed.
Tools Update Viewer and XP Users
[01:20] This potentially means that the Tools Update RC viewer (currently version 3.7.26.299443, released on March 4th) may be promoted to the de facto release viewer in week #12.
When this happens, it will obviously mean that all future builds of the official viewer will be made using the new tool chain and autobuild process. This in turn means that any Windows version of the viewer built using the new tools set (which includes MS visual Studio 2013) will not run on any version (32-, or 64-bit) version of Windows XP. To this end, the installer is being set so that it requires a minimum of Windows Vista with Service Pack 2 installed, in order for it to successfully install the viewer.
Note that this is not a deliberate attempt to block XP users from Second Life; it is purely the result of the Lab moving towards the use of up-to-date tools for building the viewer (and which will yield positive benefits elsewhere, such as with greater tool commonality between the Lab and TPVs), and some of these tools do not support windows XP due to its age and it no longer being actively supported by Microsoft.
[16:54] Some TPVs may investigate / opt to build the viewer somewhat manually using the new tool chain in such a way that it can be used on XP, but this is reportedly requires a “very large amount of work” to achieve, requires a lot of command line input, an avoidance of VS 2013, and is “really hacky”.
Project Viewers
[03:28] The Viewer-Managed Marketplace project viewer (currently version 3.7.25.298865, released on February 13th) is liable to be updated in week #12 as a result of further fixes and updates that came out of the last round of testing
[04:20] The Mesh Importer project viewer (currently version 3.7.25.298441, released on February 3rd), is currently undergoing further update with new fixes and will be updated as a project viewer in the near future.
Avatar Layers Global Limit
Vir Linden – working on the new wearable layers code
[04:41] In response to BUG-6258, “Popularity of Mesh Attachments Facilitates Need For More Alpha Layers”, the Lab is working to implement a new “global” limit to the number of system clothing layers an avatar can wear.
Under the current system, there are 12 types of clothing layers or wearables (alpha, tattoo, undershirt, shirt, jacket, underpants, pants, gloves, socks, skirts, shoes, and physics), with (generally) up to 5 of each type of wearable able to be worn at the same time, giving a maximum of 60 wearables that can worn simultaneously.
Under the new code being developed by Vir Linden, a new “global” limit of 60 wearable layers is being set per avatar, and users will be able to wear any number / combination of layers up to that limit (so you could opt to wear 60 jacket layers if you wanted, or 10 each of alpha, shirt, pants, gloves, jacket and socks, for example).
This update requires changes to both the viewer and to the server-side Appearance (SSA) service. The viewer-side changes are updates to the viewer’s logic, so it is purely checking the number of worn layers against the global limit of 60, rather than limits set for individual layers. The SSA changes will similarly support the new “global” use of clothing layers, but will also continue to support the 5-per-layer limit for viewers that do not adopt the newer code, or require a longer lead time in order to adopt the new viewer code, once it is available, thus providing a measure of “back compatibility”. The viewer code is expected to appear in a project viewer once it, and the back-end changes have cleared the Lab’s QA team.
Group Chat
[09:29] As noted in my recent updates, changes made to the group change service in the last two weeks unexpectedly resulted in up to 20% of messages failing to be delivered correctly. Simon Linden spent a fair amount of time during week #10 stabilising things and delivering further updates to try to correct the problem. As a result, in what has been called an “educational” two weeks for the Lab, the situation has been largely reversed, although some problems still remain.
The Server Beta User Group meeting on Thursday, March 12th, saw a further set of updates from Simon undergo testing on the Beta grid, and during the TPV Developer meeting on Friday, March 13th, Oz indicated that the Lab will probably undertake a further round of “serious” upgrading of all the technology associated with group chat before they declare the project in any way “finished”. This will likely involve putting the back-end service support group chat on more up-to-date hardware and OS environments.
As always, please refer to the sever deployment thread for the latest updates and information.
There was no Main (SLS) channel deployment on Tuesday, March 10th
On Wednesday, March 11th, all three RC channels should receive the same new server maintenance package comprising “internal improvements for premium users”.
When asked during the Simulator User Group meeting on Tuesday, March 10th whether the “internal improvements for premium users” was related to the change to the in-world receipt of off-line IMs, as mentioned at the last SBUG meeting, Simon Linden could only say, “I’m not supposed to announce anything, so I can’t go into details … but one thing we’re looking at this year is ways to make premium accounts better. This may or may not do something like that eventually.”
SL Viewer Updates
A new Maintenance RC viewer, version 3.7.26.299610,was released on March 6th. This includes multiple fixes and improvements as listed on the release notes and download page.
The Experience Keys viewer updated to version 3.8.0.299338 on Monday, March 9th, maintaining parity with the current release viewer.
Experience Tools
Although the Experience Tools viewer has been updated (see above), there is still no news on when Experiences might be fully deployed. In order to help build interest in Experiences a suggestion has been put forward to enable Experiences to be rated in terms of the number of people actively joining them (see BUG-6911), which could be optionally shown (at the Experience creator’s discretion in things like search listings, allowing people to judge Experiences by their popularity.
The Lab has considered allowing users to rate Experience themselves in a future update – but as point out in the JIRA comments, such a system could be open to gaming, much like the old avatar popularity ratings. BUG-6911 has been imported by the Lab, but it is currently unclear if the idea will be carried forward.
Group Chat
As also noted in my last updates, recent changes to the group chat service have seen up to a 20% failure rate in delivered messages. Simon Linden spent a fair amount of time during week #10 stabilising things once more, and notes that the situation taught the Lab more about how things might fail. He currently has a set of updates which may further improve things, and these are liable to be tested at the next Server Beta User Group meeting.
Other Items
Names Vanishing from Ban Lists
There have been reports of avatars added to a region / estate ban list or have been previously muted suddenly dropped from the list without an action on the part of the list owner. This might be connected to the old issue of bans made using radar on some older versions of v1-style viewers (notably Phoenix) failing to “stick”, or it may be something else, such as a failure to correctly update a ban / mute list.
Commenting on the subject at the Simulator User Group meeting, Simon said, “we’ve heard reports of that and have looked into it … if you ever can narrow down an instance of that happening, please note it in a JIRA … Our logs will record info about those changing but we have to know where and when to look
“I’m making wild guesses, but I think it would be either the viewer or the simulator making an update to the ban list, and somehow having bad data. Perhaps an incomplete list gets into the picture, and using that as a basis for the update it drops people. Our logs will show events like “MrNoisy was added” and “MrGoodBehaviour was removed” but finding the event is the missing part of the puzzle.
“If you have multiple regions in the estate, there’s another issue of having the changes sent out to all the regions. We’ve seen failures there and I know it’s been worked on a few times (and suspected in some of these reports).”
So, if you do encounter a situation involving an apparent ban list failure, and can log the exact circumstances / details, please consider raising a bug report.
The following notes are taken from the Server Beta User Group (SBUG) meeting held on Thursday, January 28th, 2015, and the TPV Developer meeting held on Friday, January 30th. A video of the latter is included at the end of the article (my thanks as always to North for recording it and providing it for embedding), and any time stamps contained within the following text refer to both it and the TPV Developer meeting.
Server Deployments Week 5 – Recap
On Tuesday, January 27th, the Main (SLS) channel received the server maintenance package previously deployed to Magnum and LeTigre
On Wednesday, January 27th, the RC channels should all receive a new server maintenance package comprising an Experience Keys / Tools related fix, and internal Experience Keys / Tools improvements.
It was stated during the TPV Developer meeting that this update saw the support for the Avatar hover Height (AHH) capability extend to all three RC channels, although no mention of it was made in the release notes.
SL Viewer
Tool Chain Project viewer
The Lab issued a new Tools Update project viewer, version 3.7.25.298587, on Wednesday January 28th. This viewer has been built using the new viewer tool chain for Windows and Mac builds (which includes the upgrade to Visual Studio 2013 for Windows and, Xcode (5 or 6) for Mac, and the use of the autobuild 1.0 process).
Wiki notes for using the new build tools are in the process of being written / enhanced for self-compilers – please note these are a work in progress. Notes on the new autobuild process can also be found on the wiki.
[06:42] It is anticipated that this viewer will progress through to release candidate status, quite probably in week #6 (week commencing Monday, February 2nd).
Maintenance RC Viewer Issues
Users receiving the Maintentance RC, version 3.7.25.298030, released on Tuesday, January 27th, have reported a series of irritating bugs with the viewer:
BUG-8324 (and duplicate BUG-8333) reports the appearance of the term “nolink>” in numerous places within the viewer – such as toasts and notifications – this issue has been imported by the Lab and a fix is apparently in development
BUG-8331 – some links containing the text secondlife or lindenlab incorrectly open a Place Profile in the Places panel when clicked
BUG-8328 – very jagged pictures created by the snapshot floater
BUG-1028 – the fix for MAINT-2056 “Lots of CURL GET FAILED spam in logs when browsing World Map – Beta viewer only” appears not to have fixed the problem, but rather replaced it with an equally spammy message.
Issues with this viewer can be tracked on the JIRA using this filter.
Viewer-Managed Marketplace (VMM)
[00:12] Brooke Linden was at the TPV Developer meeting to provide an update on the state of play with the Viewer-Managed Marketplace (VMM) work, which has been undergoing some beta testing with the project viewer on Aditi.
As a result of this testing, the Lab has been re-visiting the project viewer and incorporating feedback and observations for improvements into it, and are “really close” to having an updated version of the viewer available. When the appears depends on whether the Lab opt to release the updated viewer for limited beta testing on Aditi, or to wait until they’re ready to go with beta testing on the main grid, which is also not too far over the horizon.
The current plan is for more widespread beta testing to commence some time after February 14th (so as not to clash with any Valentines Day sales and promotions merchants may be running between now and Feb 14th), then run the beta for around a month, possibly longer, depending on feedback / issues. After this, VMM support will be deployed as a production tool, and things will move to a migration period which will also last around a month.
All this is still TBC, but a schedule will be posted (presumably via the Commerce / Merchant’s forum) once the Lab has worked through dates and requirements.
Experience Keys / Tools
[03:50] The Experience Keys / Tools project is currently awaiting some final server-side updates to be fully deployed (see the server release notes above). It is not anticipated that the Experience Tools viewer which is currently in the viewer release channel (version 3.8.0.298001) will require any further significant updates, so its promotion to the de facto release viewer is pending the server-side updates.
In the meantime, the Lab has already commenced work on a set of enhancement to the Experience Keys / Tools which will further extend capabilities once they are ready to be deployed.
Avatar Hover Height Project
As noted above, server-side support for the Avatar Hover Height project (see my overview for details) is now on all three RC channels, and the plan is to deploy the server code to the Main channel in week #6 (week commencing Monday, February 2nd). Because of this, the current project viewer (currently version 3.7.25.298129) is liable to be superseded by a release candidate viewer following the server-side deployments for that week.
Now in a project viewer (and soon to be a release candidate: Avatar Hover Height provides a means of adjusting your avatar’s graphical height above the ground / floor / objects, as seen by yourself and others
[18:00] A couple of small issues have been noted with AHH, and which only occur is very specific instances. One is that if an adjustment leaves your avatar floating more than around 1/2 a metre above the ground, an attempt to sit down can result in your avatar going up. In the other, standing on a prim and lowering your hover height into the prim can result in your avatar’s legs buckling as you slide into the prim. On terrain, where there can be limited physical penetration, this can be expected behaviour; but when occurring on a prim, it is suspected that the system may not be accurately assessing your avatar’s position.
[19:38] There have been one or two points of concern raised about the similarity in the name of this function – Avatar Hover Height – causing people to confuse it with the existing avatar hover slider (which will not be retired or removed from the viewer once AHH is fully deployed, as the two capabilities provide different use case solutions). The concerns are being fed back internally at the Lab, and if a name change is thought to be required, it may be slipped into the upcoming RC rebuild for the viewer, or the Lab may opt to wait and see if any confusion does arise.
There were no server deployments to either the Main (SLS) or RC channels for the week.
Upcoming Deployments
There is likely to be a server-side RC deployment in week 3 (week commencing Monday, January 12th). Details on what it will contain have yet to be finalised, however, during the Server Beta User Group (SBUG) meeting on Thursday, January 8th, Maestro Linden indicated it would contain miscellaneous fixes an improvements which will likely include:
Removal support for legacy viewer-side avatar baking
Region crossing improvements.
Simon Linden – tidying-up avatar region crossings
The region crossing improvements are for avatars only (not vehicles), and are described by Simon Linden as, “all internal and pretty minor, so please don’t get hopes up for performance improvements,” and being about “clean-up and small polishing.”
A number of regions on Aditi are running the updates – DRTSIM-273 – (Ahern, Bonifacio, Morris and Rizal), and a group test was carried out during the SBUG meeting with people walking / flying and TP’ing between these regions and between them and other regions without the updates as a further check that the changes wouldn’t result in any failures / breakage. “I’m most worried about some data subtly changing or getting lost between crossings, Simon added in this regard, following the test. “but I haven’t seen anything like that.”
Details on the updates related to legacy avatar baking will be posted with the package release notes, when available.
SL Viewer
On Monday, January 5th, the HTTP pipelining RC viewer was updated to version 3.7.24.297623, bringing it up to par with the current release viewer, and therefore matching the Experience Keys RC viewer which slipped out just before Christmas.
Experience Keys / Tools
“It’s really quite close,” Oz Linden said of the Experience Keys / Tools project at the SBUG meeting. “We’re upgrading and testing some back end infrastructure to support it; when that’s ready, we’re good to go.”
As indicated in my original overview of Experience Keys / Tools, the tools come with a number of safeguards to reduce the risk of them being used for mischief. In addition, and as a further discouragement, those wishing to use the tools to build experiences will be required to pay a fee. This has now been set by the Lab, but is not at this time being made public. Expect to hear more about it when the tools are formally released.
In the meantime, you can read more on the project in these pages using the Experience Keys tag, and those wishing to try-out beta Experiences can do so via the Seamless Experiences section of the Destination Guide. You do not need to have the Experience Keys project viewer in order to do so – although it does provide you with access to more information about any Experience you try – see my project viewer overview for details (do keep in mind that both this and the project overview linked-to above are now several months old, and certain details may have changed in the interim; I’ll have a further article on Experiences when they are launched).
Group Chat
Simon Linden is continuing to work on group chat. No major news at this time, other than he’s been carrying out further digging into why some servers seem to get “stuck” every two weeks or so and require a restart. Further data was gathered during the region crossing tests mentioned above.
Other Items
Alpha Map Support for Mesh
I first reported on this matter, which forms feature request BUG-8100 – in week 52. Since then, the idea has received a lot more feedback and further thought. However, following initial triage, the JIRA was closed by Kyle Linden on Wednesday, January 7th, with the standard, “We’ve reviewed your request and determined that it is not something we can tackle at this time.”
Inara Pey: Living in a Modemworld 