SL Projects update week 32/2: server, viewer, issues

Beeswing, Banana Island, May 2014; Inara Pey, on FlickrBeeswing, Banana Island, May 2014

Sever Deployments – Week 32 Recap

  • There was no Main (SLS) channel deployment on Tuesday August 5th
  • On Wednesday August 6th, all three RC channels received the same maintenance update, which addresses some miscellaneous bugs, and fixes the JSON issue “Valid JSON numbers like 0e0 no longer valid after 14.06.26.291532″ (BUG-6657) and includes changes from the current Main channel release.

The “fix” for the JSON issue was to in fact roll back the fix for an earlier JSON issue (BUG-6466) which appears to have triggered the more recent issue. A fix for both problems is now currently in the works.

SL Viewer

  • The Library Refresh viewer was updated on August 6th to version 3.7.14.292638 on August 6th. This viewer contains an update to a large set of libraries used by the viewer to provide security, stability and consistency improvements to this and future viewers (download and release notes)
  • As anticipated, the Zipper Viewer, intended to improve the viewer’s installation speed, has been removed from the viewer release channel. The reason for this is an incompatibility between the archive format used to zip the viewer skins for faster installation and older operating system versions.

Other Items

Maestro Linden is back from his 3-week vacation and revealed he and Caleb Linden have been working on ‘infrastructure’ updates recently, for some of the central services, which has mostly involved making sure they still work properly after a big OS update which is in the works.

HUDs Detaching / Reattaching Following Teleport

There has been a report that HUDs are acting oddly following a teleport, apparently detaching and then reattaching, but showing as “worn on invalid Attachment Point” until clicked upon. The issue seems to be primarily experienced by people using viewers with the AIS v3 updates, but it may also occur on other viewers.

These issues are somewhat similar (in part to problems previously reported with avatar attachments failing to behave as expected when using the inventory WEAR command ((see BUG-6487 and my notes here). The Lab is going to take a poke at things.

Soul Seize

There have been further reports of the “soul seize” griefing tools doing the rounds. These are objects (both in-world and possibly HUDs worn by others disguised as greeters) which offer to animate your avatar (e.g. hug you or allow you to sit), but which are intended to take control of animating your avatar.

This issue has been around a considerable time (see JIRA VWR-13228). However, a year ago in Agust 2013, the Lab deployed a partial fix for the problem. This was change to the Stop Animating Me option in the viewer (found under the Me / Avatar menu in v3 viewers), to cause it to send a message to the simulator so it revokes all animation permissions for all objects in the region.

The solution isn’t perfect – it doesn’t work where animation has been initiated by a HUD, and at the time the update was made, concerns were raised that griefer might work around it. Whether these latest reports refer to a new flavour of the tool which does get around Stop Animating Me update is unclear.

SL project updates week 32/1: server, viewer

The Snow Lion, Oceanside dAlliez; Inara Pey, May 2014, on FlickrThe Snow Lion, Oceanside dAlliez (click for full size – blog post here)

Server Deployments – Week 32

As always, please refer to the server deployment thread for the latest updates and status on deployments.

  • There was no Main (SLS) channel deployment on Tuesday August 5th
  • On Wednesday August 6th, all three RC channels should receive the same maintenance update, which addresses some miscellaneous bugs, and fixes the JSON issue “Valid JSON numbers like 0e0 no longer valid after 14.06.26.291532” (BUG-6657) and includes changes from the current Main channel release.

SL Server

Release Viewer

The de facto release viewer was updated on Monday August 4th to version 3.7.13.292225, formerly the Group Ban RC viewer. For an overview of the group ban functionality, please refer to my article here (note this is based on the project viewer, although the functionality has not changed significantly).

All other viewer in the release and project channels remain as per my Current Viewer Releases page.

Oculus Rift Viewer and Oculus DK2

Versions of the Oculus Rift DK2 kit are beginning to appear, and those who have received the updated headset are noting that the current Oculus Rift project viewer (version 3.7.12.292141 – see the Alternate Viewers wiki page) is not compatible with the newer equipment, particularly with regards to the updated tracking system and the new display modes (see: RIFT-130).

Given the extent of changes between the DK1 and DK2 headsets, incompatibilities shouldn’t be that surprising, and problems are limited to SL; other applications built for the DK1 are also apparently having their share of niggles with the newer hardware. At the time of writing, it wasn’t clear if the Lab had or had not received their Oculus DK2, so it might still be a while before an updated version of the viewer with DK2 support appears.

User Group Meetings, Week 33

There will be no Open-source Developer meeting or Simulator User Group meeting on Monday August 11th or Tuesday August 12th respectively. This is because the Second Life technical team will be involved in a strategy  / team building get-together in the physical world, and so will all be offline for in-world meetings as a result of travelling, etc.

Other Items

Off-line E-mail Issues

There are reports that offline IMs to e-mail are being blocked by certain ISP, most notably United Internet in Germany, which operates a number of web-based e-mail services, including GMX, 1 and 1, web.de mail.com and A1. The problems started in early July (see this forum thread and BUG-6591 and BUG-6717), and was initially seen as an issue within SL.

However, SL user MartinRJ Fayray, himself based in Germany, contacted 1 and 1 and received confirmation that certain IPs from SL have been blocked on account of the amount of “spam” they are generating.

As the blocking appears to be simulator IP-based, the situation has given rise to some confusion, as some offline e-mails do get through to users, and others don’t (depending on where they were initiated within SL), leading to people viewing the problem as Lab-based issue.

Things currently appear to be up in the air as to what might happen, as unblocking IP ranges is something United Internet needs to do, although United Internet appear to be waiting for the Lab to contact them.

This is not the first time issues with IM to e-mail have been encountered. In 2013, many Gmail users found that their off-line e-mails were no longer being received, although that was due to a change in Gmail’s filtering policies which could be rectified by a settings change at the user’s end.

SL projects update week 31/2: viewer, group chat

Matoluta Sanctuary, Sartre; Inara Pey, July 2014, on FlickrMatoluta Sanctuary, Sartre, July 2013 (Flickr)

Server Deployments Week 31 – Recap

  • On Tuesday July 29th, the Main channel was updated with remaining recent feature changes and bug fixes previously deployed to the RC channels – release notes
  • There were no RC deployments.

SL Viewer

The Zipper viewer, offering a faster install, reappeared on Wednesday July 30th, after vanishing from the Alternate Viewers wiki page in May. There are apparently an issue with the XUI Preview Tool being broken, which has now been resolved.

The new version of the viewer – 3.7.13.292263 – appeared as a release candidate in the release viewer channel, rather than a project viewer, where it resides with the group ban viewer and the library refresh viewer, both of which were updated in week 30, are likely the strongest candidates for promotion as the next de facto release viewer.

Group Chat

Testing of on-going group chat updates took place during the Server Beta meeting on Thursday July 31st.

Simon Linden is once more digging into the group chat code
Simon Linden is once more digging into the group chat code

As noted in a previous report, one of the major causes of issues with group chat lies not with the actual messages being sent back and forth, but rather as the chat server tracks who in online or not. The server maintains a list of who is online and in the group chat at a given moment, and is constantly updating the list as people join / leave the session; these updates are then sent to everyone else still active in the group, which interferes with the sending / receiving of actual messages.

“Imagine a popular group with, say 120 people online,” Simon said during the meeting. “Let’s guess the average online time is an hour … and that number varies widely, as there are a LOT of people who are connected for only a minute or two, maybe just checking IMs, see who’s online, or trying to fix something. But with 120 people … that’s very roughly an update every 30 seconds [14,400 updates an hour], sent to the whole group.”

Not only does this impact the sending / receiving of chat messages within the group, it can also impact other group chat sessions which are running on the same back-end server, as they are being starved of resources.

The code being tested on July 31st had been set to delay the sending of these updates from the server in order to see if it improves the throughput of actual messages. The downside of this is that the member list updates are somewhat delayed; however, this would seem to be a small price to pay in order for an increase in the reliability of messages actually getting through the system. As it is, the delay is configurable, so Simon was gathering data to see how the updated code works in terms of people joining / leaving chat sessions and sending messages. The results are liable to be known next week.

One possible future option for group chat is for people to be offered the ability to opt-in or out of receiving group chats until such time as they join a group chat (some TPVs already have an option to disable group chats until such time as you opt to join them).While this may help with the “I’m here!” messages sent to all groups on log-in, and which exacerbate the problem somewhat (again as described in the update linked-to above), such an approach is not seen as optimal, as it is possible users won’t change their behaviour, but will simply opt-in to all group chat sessions anyway.

Simon has also been tracking down an odd bug with joining a group and being able to open a chat session with it. “It’s really an odd one where opening the group is very slow or times out,” he said at the July 31st meeting, “and then can be immediate the next time you try. From what I can tell the chat server isn’t getting the messages … so somewhere between the viewer, simulator and chat server it gets lost.”

 Other Items

BUG-6736 is a feature request for the updating or removal of the current limit on the distance at which objects can be linked (see linkablility rules). The advantage in increasing the limit is that it could allow for bigger builds (in terms of footprint) without having to rely on scripted rezzing systems. A problem here is the if increased, there is a risk that the ability to link objects over greater distances might cause issues were said distances are close to or exceed the draw distance / interest list distance.

“You will get some really funky update issues if the link size is larger,” Simon said at the Server Beta meeting. “As soon as it gets close to your draw distance, things go bad, as in … you stumble against something you can see.”

Commenting on this, Lucia Nightfire added, “I noticed a selection bug/gripe with multi selection and one prim being out of interest range on rez, you deselect everything by clicking on something else then if you pull your cam back you magically select stuff that was out of your interest range.”

Responding to this, Simon said, “Yeah, that’s the kind of thing that can get confusing, you won’t see what you expect because the root might be farther away than your draw distance … That said, I understand the builder desire to make larger parts, but those limits are there because it can conflict with the interest list logic about your updates.” As such, it would seem unlikely that there will be much in the way of change to the linkability limit.

SL projects update week 31/1: server, group chat

The Lab's Battery Street staff (image: Ebbe Altberg, via Twitter)
The Lab’s Battery Street staff (image: Ebbe Altberg, via Twitter)

Server Deployments Week 31

As always, please refer to the server deployment thread for the latest news and updates.

Main (SLS) Channel

On Tuesday July 29th, the Main channel was updated with remaining recent feature changes and bug fixes previously deployed to the RC channels. Namely:

  • Fix for “Temp Attachments are sometimes not removed on the viewer when detached from a region change event.”
  • L$ transactions from llTransferLindenDollars or llGiveMoney now include a “Description” field with the object’s name in the L$ transaction history section of the website.

See the release notes for further information.

Release Candidate Channels

There will be no Release Candidate channel deployments in week 31. This means there is unlikely to be any main channel deployment in week 32 (week commencing Monday August 4th), although RC channel deployments are anticipated for that week.

Experience Keys

There is a little more news on the Experience Keys beta, which Oz Linden, attending the Simulator User Group meeting on Tuesday July 29th,  indicated was going well. A possible extension to the beta had been sort-of hinted at in previous meetings, but Oz indicated that whether this will or won’t be the case has yet to be decided.

It was also indicated at the meeting that the idea fees which may be associated with the use of Experience Keys by creators may also have had further discussion within the Lab.

The idea of charging creators a nominal fee for the use of Experience Keys was first discussed at the Experience Keys preview during a TPV Developer meeting. A nominal fee or some restriction on accessing the tools (e.g. such as having to be a Premium member) was seen as a means of perhaps discouraging those who might otherwise use the capabilities with intent to grief harass others. While at the time it was admittedly an exploit of the original Advanced Content creation permissions system, the June 2012 incident did demonstrate how someone with access to the tools might use them to cause upset.

Commenting on the likely charges (if any) which may be levied and whether they have been agreed internally, Oz Linden said at the Simulator User Group meeting on Tuesday July 29th, “We don’t discuss pricing here… not our area. I believe they have, but I don’t know details.” Simon Linden then added, “We can’t talk about any details like that until they’re officially announced.”

llSetLinkAlpha Updates Issue

BUG-1786 has been around for over a year, and takes the form of multiple prims in a linkset randomly failing to update correct when using  llSetLinkAlpha. For example, you have a cube made up of a number of smaller cubes so that when clicked, the entire linkset should switch from solid to transparent or back again, but only some of the prims in the linkset appear to change on being clicked. with some remaining solid when they should be transparent. The problem has been reported as being more noticeable on linksets of 34 or more prims, but has also been reported on smaller linksets.

The problem seemed to first occur with the 26th February 2013 server deployment 13.02.15.270481, and the issue has been thought to be server-side. However, the problem seems to be more prevalent with lower bandwidth settings (e.g. 500 or lower – 500 being the SL viewer default), and can vanish with higher bandwidth settings (e.g. around 1200-1500), suggesting the issue might be related to packet loss, particularly if there are a lot of simultaneous updates going on, or a possible race condition within the viewer.  A sample object has been passed to the Lab to let them poke at things further.

Group Chat

The group chat test which suffered cancellation at the July 24th Server Beta meeting may get to be run at the upcoming meeting on Thursday, July 31st.

The Next Simulator UG Meetings

Simon Linden is on vacation next week, and may not be able to attend the week 33 Simulator UG meeting either, due to other commitments. He’s hoping to get someone at the Lab to cover for him at both meetings.

SL projects update 30/2: server, viewer, group chat

The Bayou, April 2014; Inara Pey, May 2014, on FlickrThe Bayou, April 2014 (Flickr)

Server Deployments – Week 30 Recap

  • On Tuesday July 22nd, the Main channel was updated with the infrastructure project deployed to the Magnum RC in week 29, and which adds support for the upcoming changes to the Skill Gaming policy, and includes the updates previously on LeTigre and BlueSteel- release notes
  • On Thursday July 24th, all three RC  channel were updated with the infrastructure support for the upcoming changes to the Skill Gaming policy, and the updates previously deployed to LeTigre and BlueSteel – release notes.

SL Viewer

The Library Refresh viewer was updated to release candidate status with the release of version 3.7.13.292194 on July 23rd. This viewer contains an update to a large set of the libraries used by the viewer to provide security, stability and consistency improvements to this and future viewers.

Group Chat

Simon Linden: continuing to work on group chat issues
Simon Linden: continuing to work on group chat issues

The anticipated group chat test didn’t materialise at the Server Beta meeting on Thursday July 24th as a result of Simon Linden coming across a last-minute issue which needed to be resolved ahead of further tests. He and Oz did, however, explain some recent discoveries within the chat system.

“While the earlier update to group chat didn’t give us any significant performance boost, we got a lot more information out of the servers,” Simon said. “And what we found was a big part of the group chat system load is not the chat messages you care about, but the updates to who is in the session or not.”

Oz added, “Those updates happen whether or not you’re displaying who’s in the session, in every group you’re in.”

Simon continued, “You can actually see this in the viewer if you add a line of code to log something whenever an update comes in to tell you who’s in the group chat … you’ll be surprised how many you get. The load goes up as the group size goes up … with a larger group, people are joining and leaving more often, and there are more people to update.”

People joining / leaving a session are recorded by the chat server. “It has a list of who’s online and in the group chat at that moment,” Simon explained, “it’s adding and removing from that list, and [generating] the resulting updates, that are the problem, [causing them] to be sent to everyone else still active in the group as they do so.”

The growth curve of these updates is described as exponential, and there is a knock-on effect with them as well; as group chat sessions share server resources, it is possible that a large group chat session, with multiple users joining  / leaving it and thus causing it to generate lots of updates can affect other group chat sessions hosted on the same group chat server, slowing them down as well.

While the chat servers are due for a hardware change, which is hoped will improve performance to a degree, simply adding more hardware to the chat service back-end isn’t seen as a solution, as it’s the exponential manner in which the updates grow which needs to be reduced and controlled. The testing Simon had hoped to run during the Server Beta meeting was to test some improvements he had been making to the queuing of the updates and in combining messages to hopefully reduced the load. However, in running over the code, he encountered a glitch that he needs to resolve before the testing can proceed.

Another issue with the group chat system is that when users log-in to a Second Life session, they automatically join all 42 of their groups, sending a “I’m here!” message to all 42 groups so that they can start receiving messages from active groups. This has obviously been exacerbated each time the limit on how many groups a person can join has been raised, so as Oz pointed-out during the meeting, “upping it again would make it even worse, so until this is dealt with, don’t even ask… 🙂 .”

HTTP Updates

As indicated by Monty Linden at the last TPV meeting, there are further server-side updates which should further assist with improvements to texture and mesh asset downloads. These are now nearing the point where they are likely to be surfacing (although quite when isn’t clear), prompting Oz to comment, “We’re setting up some experiments with server side changes that will complement the pipeline viewer, but are not strictly speaking dependent on it. When we’re confident that our test setup is ready, including how to measure the results, we’ll invite you folks to help us test.”

 

SL projects update week 30/1: server, viewer

Server Deployments – Week 30

As always, please refer to the server deployment thread in the forums for the latest information and updates.

Main (SLS) Channel

On Tuesday July 22nd, the Main channel was updated with the infrastructure project deployed to the Magnum RC in week 29, and which adds support for the upcoming changes to the Skill Gaming policy, and includes the update to improve L$ transaction logging for payments made by scripted objects previously on LeTigre and BlueSteel- release notes.

Release Candidate Channels

On Thursday July 24th, all three RC  channel should be updated with the infrastructure support for the upcoming changes to the Skill Gaming policy, and includes the L$ transaction logging update and Temp attachment fix previously deployed to LeTigre and BlueSteel.

SL Viewer

On Tuesday July 22nd, the de facto release viewer was updated to version 3.7.12.291824, previously the Maintenance RC release candidate. This viewer includes over 40 MAINT fixes, including:

  • MAINT-3135 Cocoa Viewer: Mac: Maximizing the viewer leaves garbage on the screen
  • MAINT-3171 Alt-clicking while moving mouse can move the camera significantly
  • MAINT-2980 Reevaluate the 512 meg texture cap
  • MAINT-4216 Double clicking on anything in COF removes it from your avatar – including skin, shape, hairbase and eyes – results in bakefailed avatar
  • MAINT-4001 Received Folder is movable within Recent Tab – see my notes here on this issue
  • MAINT-3610 SL viewer partly ‘eats’ chat-message.

Tuesday July 22nd also saw two other viwer updates:

  • Tthe Group Ban RC viewer updated to version 3.7.13.292225. This allows group owners and those assigned to a specific role to ban (and eject) others from joining the group. Useful for dealing with spammers and other trouble-makers in open enrollment groups. For an overview of the group ban viewer, please refer to my preview notes
  • The Oculus Rift project viewer also updated to version 3.7.12.292141, to bring it up to parity with the release viewer 3.7.12 code base. Note at the time of writing, there were no supporting release notes for this update – please refer to the Alternate Viewers wiki page for the downloads.

Group Chat

Simon Linden is planning on a further group chat test at the Server Beta User Group meeting on Thursday July 24th. Commenting on this at the Simulator UG meeting on Tuesday July 22nd, he said, “those have been really helpful with our development.   The more people who can come, the better. The test is usually at the end, a little before 4:00 if we don’t run out of other topics.”