Important note: The SL Go service is to be shut down on April 30th, 2015. For more information, please read this report.
As I (among others) reported on Tuesday, October 14th, OnLive released a version of their SL Go service for the iPad.
At the time of the release, there was an unexpected rotation issue which meant that the screen orientation was locked in one horizontal orientation – if you flipped the iPad around, the screen would appear upside down, although the input areas might actually flip with the screen but still be displayed as if they were upside down.
This was a particular nuisance for people using a bluetooth keyboard or docking station with their iPad, as it meant the viewer would be displayed upside down when docked with either.
On Friday, October 24th, Dennis Harper from OnLive dropped me a note to inform me that this rotation issue has now been corrected, The app should update for those who already have it, and the fix is now a part of all downloads obtained via the Apple iTunes store.
On Tuesday, October 21st, the Main channel was updated with the server maintenance release previously deployed to the LeTigre and Magnum RCs in week 42. This update includes a crash fix and improves the delivery pipeline for abuse reports
On Wednesday, October 22nd, the release candidate channels were updated as follows:
BlueSteel and Snack will remain on the CDN project, and were updated with the same server maintenance package being deployed to the Main channel
LeTigre and Magnum were both updated to the CDN for texture and mesh fetching.
Upcoming Deployments
There is a further server maintenance package being prepared for deployment. however, it is still undergoing testing, and it is not clear whether it will be ready for deployment on Wednesday.
CDN and the Future of Snack
With the CDN now supporting texture and mesh fetching across all of the primary RC channels, there is a chance it may be promoted to the Main channel in week 4, particularly given that, outside of CDN support, all the server channels are now in parity. However, the Lab is making no commitments to this, and is taking a “wait and see” approach on what may happen. In particular, the Ops team will be monitoring performance over the weekend, when SL usage is expected to be at its busiest since the RC-wide deployment of CDN support, to see how things hold up.
As it stands, there are an estimated 130 regions on the Snack RC. As these all came from the Main channel, they will likely be switched back to that channel once it has CDN support, and Snack will again be dissolved until it is once again required.
Group Chat
With the most recent changes to the back-end servers, Simon Linden is continuing to poke at the code to see what can be improved. He’s now looking at the code that routes the messages – where it goes to find where group members are on the grid in order to be able to deliver messages to them, with the aim of trying to improve things there.
At the moment, if you move between regions which active in a group chat, the first message to be sent to you after you move will be sent to the region where you were and generate a failure message which has to go back to the chat service, which they has to relocate you and then attempt a further delivery. It is hoped that this can be improved, and there was a brief test following the Server Beta User Group meeting on Thursday, October 23rd to this end, just to see how the code is performing.
It has been suggested that the Lab should bypass the region hosts altogether when handling group chat, and attempt to route it directly from the chat servers to the viewer. However, this would likely require some serious re-engineering; more to the point, it could introduce security vulnerabilities.
Other Items
Aditi L$ Balance
A question was asked on how account L$ balances are updated on Aditi. Maestro replied that account balances are checked nightly (SLT), and any falling below L$ 10K (because someone has been uploading mesh models for testing / viewing prior to paying to upload them on Agni, for example), they are automatically increased by L$5K. However, if someone resets their account password (which triggers a reset on Aditi), their L$ balance will change to match their L$ balance on Agni, although again, if this is below L$10K it should be topped-up but L$ 5K overnight.
On Tuesday, October 21st, the Main channel was updated with the server maintenance release previously deployed to the LeTigre and Magnum RCs in week 42. This update includes a crash fix and improves the delivery pipeline for abuse reports.
RC Channels
On Wednesday, October 22nd, the release candidate channels should be updated as follows:
BlueSteel and Snack will remain on the CDN project, but should also be updated with the same server maintenance package being deployed to the Main channel
LeTigre and Magnum should both be updated to the CDN for texture and mesh fetching.
SL Viewer
On Wednesday, October 15th, the Lab issued a Grid Status update on potential problems accessing external websites using the viewer’s internal browser. The notice was issued as a result of the Padding Oracle On Downgraded Legacy Encryption (Poodle) vulnerability reported by Google.
At the time the notice was released, the Lab indicated they were working to fix the issue (via the removal of SSL v3.0 support within the viewer’s browser). This work was completed at the end of week 42, and resulted in the release of the Browser Fix RC viewer, version 3.7.18.295539.
On Monday, October 20th, the Browser Fix viewer was promoted to the de factorelease viewer. Further information on it and the POODLE vulnerability can be found in a separate article in this blog.
MAC Yosemite Issues
There have been some mixed results for those Mac users updating to the 10.10 Yosemite version of OS X, which reached a consumer release status on October 16th.
While most of those updating have found it relatively smooth, there has been a report of keyboard shortcuts borking, although there is a workaround should others encounter the same problem – see here.
A potentially more widespread issue people may encounter is that with Yosemite, Apple have apparently reverted to older nVidia drivers (as reported by Cinder Roxley at the Open-source Developer’s meeting on Monday, October 20th). One JIRA on the matter has already been filed (see BUG-7575), although if Apple have chosen to roll back some / all of their default drivers, there’s not a lot the Lab can do about matters. Cinder also reports that Yosemite makes greater use of OpenGL, which she indicates can tax GPUs further.
CDN
Following the scheduled Wednesday deployment, all of the RCs should be using the CDN. This represents around 30% of the total grid, possibly a little more, given the existence of Snack. The BlueSteel deployment does not seem to have resulted in the kind of performance issues experienced during the expansion of Snack to 250 regions, suggesting the latter problems were most probably down to overloading the sim hosts with too many high volume (in terms of avatar numbers) regions. If all goes well in the coming week, it is likely that the Main channel will gain CDN support in week 44.
Group Chat
“Today we also finished an update to the group chat servers,” Simon Linden announced at the Simulator User Group meeting on Tuesday, October 21st. He continued:
Many of you have helped us test that at Thursday’s beta user group, and that’s now on the newest version. Those are back-end servers … when you send something to group chat, it goes from the viewer to the region you’re on, then to the (new) back-end chat servers, and from there it’s distributed to everyone in the group.
While no promises are being made as to the overall improvement – or how visible improvements may be to users, Simon went on to note, “I know [from] looking at the metrics and data we can gather … the systems are running much nicer than before.”
As well as working on the update issues (see my previous group chat updates), the Lab has been brainstorming ideas on how groups might be configured differently. There are a lot of groups that have very heavy chat usage, and others where chat is actively discouraged in favour of outgoing notices; however the latter are handled exactly the same as those where chatting goes on, generating updates as people log-in and out, etc., even though these aren’t actually required. Exactly what might be done is unclear at this time, but it would appear ideas are being put forward and discussed.
Updates for the week ending: Sunday October 19th, 2014
This summary is published every Monday and is a list of SL viewer / client releases (official and TPV) made during the previous week. When reading it, please note:
It is based on my Current Viewer Releases Page, a list of all Second Life viewers and clients that are in popular use (and of which I am aware), and which are recognised as adhering to the TPV Policy. This page includes comprehensive links to download pages, blog notes, release notes, etc., as well as links to any / all reviews of specific viewers / clients made within this blog
By its nature, this summary presented here will always be in arrears, please refer to the Current Viewer Release Page for more up-to-date information
Browser Fix RC viewer version 3.7.18.295539 released on October 17th – core updates – SSL 3.0 removal from internal browser due to the POODLE vulnerability (download and release notes)
HTTP Pipelining RC viewer version 3.7.18.295372 released on October 15th – core updates: Pipelined HTTP Operations for Mesh and Texture Fetches; Inventory Fetch Improvements (download and release notes)
Maintenance RC viewer version 3.7.17.294943 – withdrawn from the release channel due to significant issues and a range of attachment bugs which affect it (see my week 41 update from the TPV developer meeting, under “AIS v3 Issues”)
Project viewers:
Oculus Rift project viewer updated to version 3.7.18.295296 on October 13th – core updates: Oculus Rift DK2 support (download and release notes)
Cool Viewer Stable Branch updated to version 1.26.12.21 and Cool Viewer Legacy Branch updated to version 1.26.8.79, both on October 18th – core updates: please refer to the release notes.
Unless indicated otherwise, the notes below are taken from the Server Beta meeting held on Thursday October 16th, 2014. The transcript is available here, and the support agenda notes here.
There was no code promotion to the Main (SLS) channel on Tuesday, October 14th – this follows from there having been no deployments to the primary RC channels in week 41
On Wednesday, October 15th, the primary RC channels were updated as follows:
Bluesteel received the CDN texture & mesh fetching capabilities – release notes
LeTigre and Magnum both received a new server maintenance package, which includes a crash fix and improves the delivery pipeline for abuse reports.
SL Viewer
HTTP Pipelining RC
On Wednesday October 15th, the Lab released the HTTP Pipelining RC viewer. Version 3.7.18.295372 brings with it:
Pipelined HTTP Operations for Mesh and Texture Fetches: this feature allows the viewer to issue multiple asset fetches on a connection without waiting for responses to earlier requests. This reduces the impact of a user’s physical location on scene loading and generally improves the experience for everyone
Inventory Fetch Improvements: Inventory folder and item fetches are getting some of the same treatment that textures and meshes did in previous releases. Initial inventory load should be quicker and more robust for all users.
A blog post from Monty Linden accompanies the release, which provides further information on the HTTP project alongside of the viewer release notes.
A blog post from Monty Linden arrived with the Pipelining RC viewer, and explain how the viewer (and CDN) should generate further improvements in texture and mesh fetching operations
Maintenance RC Viewer Removal
The latest Maintenance RC viewer, version 3.7.17.294943, has been withdrawn from the release channel due to significant issues and a range of attachment bugs which affect it (see my week 41 update from the TPV developer meeting, under “AIS v3 Issues”).
CDN
As noted above, use of the CDN service for texture and mesh data was extended to the BlueSteel RC in week 42, and there is some anticipation within LL that it could be deployed to all three of the primary RCs sooner rather than later.
An issue has been noted on some regions running on the Snack RC where people are seeing very slow parcel information updates (e.g. the name of the parcel taking up to a full minute to appear in the status bar, for music stream changes to occur, etc.).
The thinking from the Lab is that these problems are a result of the Snack sim hosts receiving a top-heavy load of regions with high traffic counts, resulting in possible resource contentions with certain data lookups as a result – contentions which may have actually brought the regions down had it not been for the CDN, as Maestro Linden commented:
In the cases I’ve seen so far, the slowness was due to a rapid rate of requests from all the users logged into the regions. If the sim were serving all the texture and mesh requests as it traditionally would, the regions would have ‘fallen over’ before becoming that populated.
So this would seem to be an unintentional cloud with its own silver(ish) lining (at least the regions didn’t fall over). Nevertheless, the issue isn’t welcome. Given that BlueSteel has a more even distribution of regions across sim hosts (e.g. the servers are not all packed to the gills with high volume traffic regions), It is hoped that the issue will not exhibit itself there. Checks (either by the Lab or residents or both) are likely to be carried out on busy regions on the RC to see if the problem does raise its head, which might indicate some further investigations might be required.
Assuming all goes well with the BlueSteel deployment, it is thought that CDN support will likely extended to the remaining RCs sooner rather than later. In the meantime, for those interested in seeing how use of the CDN works for them, a list of regions by RC can be obtained via Tyche Shepherd’s Grid Survey website, allowing BlueSteel regions to be easily located. Just click the RC Server Regions button (note the list will require a little cleaning-up, post download).
On Wednesday October 15th, Monty Linden blogged about his HTTP work and the CDN, using the rather unusual title, The Sky Over Berlin (and Elsewhere). It’s a great piece of reading, although I can’t help thinking that Monty’s sign-off at this end of it would have perhaps suited the subject matter far better: nous sommes embarqués – “adventure is ours for the taking”!
The first part of the post recaps on Monty’s initial work in improving Second Life via the HTTP project. This started as far back as mid-2012, with the first pass focused on improving the mechanism by which textures could be obtained for rendering via HTTP, which entered widespread use around November 2012, with the release of the 3.4.3267135 viewer.
This work was followed by Monty labouring to improve mesh fetching as well, and to improve the overall reliability of HTTP, which I blogged about in March 2013.
At the start of 2014, Monty blogged on his work up to that point, and looked ahead to future activities. As a part of the post, he included a graph showing how the work carried out up to that point improved texture and mesh request handling.
In January 2014, Monty blogged about his HTTP work, and indicated how the work had raised the request rate ceiling within the viewer for texture and mesh data from A up to the blue line of C
In his latest post, Monty picks-up where his January post left off, demonstrating how more recent improvements are starting to improve things further – notably through the use of HTTP pipelining (the release candidate viewer for which has now been issued – see below), and the ongoing deployment of the Content Delivery Network service for texture and mesh data delivery.
In his latest blog post, Monty indicates how both HTTP pipelining (the “post 3.7.16” markers, denoting the introduction of the pipelining viewer) and the use of the Highwinds CDN service (denoted by the DRTSIM-258 markers) should further help improve data transmissions and performance
All told, Monty’s work has been a remarkable undertaking which benefits Second Life enormously, and helps to set the path towards possible further improvements in the future. As such, he really is one of the heroes of Second Life and the Lab.
HTTP Pipelining RC Viewer
The HTTP Pipelining viewer was issued as a release candidate viewer shortly after Monty’s post went to press.
Version 3.7.18.295372 enables the viewer to issue multiple asset fetches on a connection without waiting for responses to earlier requests. This should help inprove things like initial scene loading quite aside from any additional benefits gained through the CDN deployment work. In addition, the viewer includes improvements to inventory fetching, as Monty noted in his blog post:
The HTTP Project has focused on textures and meshes. But the inventory system, which maintains item ownership, is often described as… sluggish. So as an exercise in expanding the use of the new HTTP library, the pipelining viewer was modified to use it for inventory fetches. As with textures and meshes before, inventory is now fetching in the ‘C’ region of its specific performance graph. The difference can be surprising.
Having had the opportunity to test the pipelining viewer somewhat prior to its appearance as an RC, I can attest to this. While I keep my “active” inventory to a modest size (around 10,000 items unpacked, the rest boxed until needed), I found that an inventory download with a cleared inventory cache (nothing saved on my computer) averaged 9-10 seconds using the pipelining viewer, compared with an average of 2 minutes 50 seconds to 3 minutes with the current release viewer (3.7.17.294959). Whirly Fizzle, using a 105K inventory had even more impressive results: with a cleared cache, her inventory loaded in under 3 minutes on the pipelining viewer, compared to 16-18 minutes on the release viewer.
The release notes for the viewer contain additional information about the updates, again written by Monty, and these make worthwhile reading alongside of his blog post.
Important note: The SL Go service is to be shut down on April 30th, 2015. For more information, please read this report.
Inara Pey: Living in a Modemworld 