HTTP updates: the what, why, who

Update March 27th: Commenting on the open-source development mailing list, Monty Linden states: “It looks like Beta (Aditi test regions) will be wrapped up shortly. If you’ve wanted to try these out but haven’t yet, now would be a good time to jump in there.” 

Linden Lab is in the process of making a number of improvements to Second Life which should benefit both the platform and users. Once deployed, some of these updates will be clearly visible as they gain widespread use in-world, such as the upcoming materials processing capabilities. Others will be perhaps more noticeable because they require a viewer update – as is the case with server-side baking, rather than being obviously visible in everyday use. Some will have more of a “background” impact, rather than anything which is clearly visible in-world (although they may make their presence felt for the more keen-eyed).

Among the latter category of changes are the HTTP updates currently being tested on Aditi and which will soon be popping-up on a Release Candidate channel. This work is being spearheaded by Monty Linden, and has been under development as a part of the Shining Project initiative kicked-off by Linden Lab in 2012.

Several of my SL projects update reports have covered Monty’s work, and will continue to do so in the future. The aim of this article is to bring the various threads together in a single post, in order to provide a  broad overview of what it all means without getting caught-up in the technical minutiae.

Communications between the viewer and the SL servers are subject to many vagaries. Network issues can occur locally (i.e. with a user’s own network), or at the ISP level, for example, long before they actually involve the SL servers. There is little LL – or the support team for whatever viewer is used to connection to SL  – can do in these instances.

However, network issues aside, there is much work that can be done to improve viewer / server communications and make connectivity between the two more robust – and this is the focus of Monty’s work. Some of this has to do with gradually switching aspects of the service away from the older UDP services within SL to HTTP-based services, and some of it has to do with improving the existing HTTP services employed by SL and making them both more robust and (hopefully) a little easier on older models of routers.

Initial Work

As mentioned above, Monty’s work is encapsulated within the Shining Project, and is being carried out in a number of phases. The first phase of this work was actually completed during the second half of 2012, and focused on improving the HTTP texture fetch mechanism both server-side and within the viewer by which textures are obtained for rendering. This work started to go into widespread use around  November 2012, when the viewer code was made available and Linden Lab announced the new capability thus:

A new scheme for performing HTTP operations is introduced with this release. It is intended to reduce crashes and stalls while performing HTTP operations and generally enable performance and reliability improvements in the future. In this release, it is being used by the viewer’s texture retrieval code. Our expectation is that it will provide consistent and predictable downloading of textures.

Following the release of the viewer code, many reported they were seeing significant improvements in texture downloads, and a resultant improvement in texture rendering.

As a part of this initial work, Monty also started examining connectivity between the server and the viewer (number of actual connections opened, etc), and found that it can cause significant hardships for older classes of router, many of which incorporate a firmware-controlled “lock-out” which can be triggered when too many connections are opened when using HTTP, and so can cause users issues (hence the recommendation which some support teams give to disable HTTP textures within the viewer if connection issues are being experienced).

Second Phase

At the start of 2013, Monty commenced work on the second phase of the project, which is currently focused on the server-side of things (that is, there are currently no viewer-side code changes). In particular he is looking at further improving texture and mesh asset-fetching from the server and at implementing HTTP persistent / keepalive connections capabilities, which should enhance the overall robustness of such communications (some of which may hopefully see some connectivity improvements for those people using older model routers, as noted above).

Work on the second phase of the project has been ongoing for a while, with Monty carrying out a lot of internal testing (over a month with the LL QA team). Recently, the work moved to more open testing on tha beta (Aditi) grid, with Monty setting up three test channels of three regions apiece in order to test the new capabilities. Precise details of these test channels and regions can be found in my week 11 SL project update. Testing can involve anyone, and obviously, those with a technical understanding of HTTP protocols and / or server-viewer communications are particularly welcomed to test the capabilities and report back on their findings . Monty is obviously keen that the following people try the services:

• TPV developers
• Those who are using older model routers
• Those who provide external services for Second Life wich communicate with in-world objects via HTTP (e.g. combat systems, vendor systems etc)

With regards to the last point, that of external services, Monty is keen to seen their involvement in testing because while the HTTP-IN service (which uses llRequestURL and llRequestSecureURL) remains unchanged with this phase of the work, the path it does transit the new capabilities he is developing, and so may be affected.

So, if you have created a combat system or a vending system or similar which utilises HTTP to communicate between in-world objects and external service or web page, you may want to make sure you conduct tests on the test regions on Aditi – the sandbox regions with their more open access have been established to assist with such testing. Any issues with the services should be reported in a Bug Tracker JIRA.

Similarly, if you are using an older router model and have experienced connection issues in the past when using HTTP, you might want to visit the three test channels on Aditi and see how your router manages with each of them with things like HTTP texture-fetch enabled. Again, issues with disconnection, etc., should be reported via the Bug Tracker, and remember to include details on your viewer / location, which can be obtained from Help > About [viewer name].

Current Status and the Future

There is no current time frame as to when phase two of Monty’s work will actually reach the main grid; there is a lot going on right now on the server-side of things, particularly when the Shining project is taken as a whole. Much may also hinge on how well the new capabilities scale as they are deployed across the grid.

Assuming all goes well with the second phase of the work, Monty has a number of other elements he’d like to take a look at in the future. These include further improvements to the mesh handling side of things, improvements around the HTTP-In service, and even deeper work – such as addressing weaknesses within libcurl, upon which SL’s HTTP services are built and which Monty views as not having a particularly good model for connection management. As such, there is liable to be a lot more to come on the subject of HTTP changes.

Also in the longer term, and once he is confident that the new HTTP capabilities are offering a better, more reliable service for the majority of users than their older UDP counterparts, Monty hopes to be able to gradually remove the UDP code from the platform.

However, this is liable to be a protracted process. As Monty stated, when initially discussing his plans in public on Friday January 4th, “The UDP assumption is baked into a lot of failover paths and hidden mechanisms, it’s all over the place. So I’m always nervous about removing it blindly; but, when the opportunity arises, yup, we’re going to look at that question.” There may also be questions around the HTTP being suitable for use within certain countries; some users in Japan have indicated that UDP prehaps provides them with a more reliable service than HTTP. Whether this latter point prove to have any major impact on future work remains to be seen.

Nor, it should be pointed out, is Monty’s work the only aspect of HTTP changes. The upcoming server-side baking / Avatar appearance project exclusively uses HTTP viewer / server communications, and Baker Linden’s work on improving group services (large groups management, etc), also involved a move from UDP to HTTP.

In the meantime, Monty’s work is progressed on a step-by-step basis, and further updates will appear in this blog as and when they are given.

Related Links

• Project Shining announcement
• Overview of HTTP persisent/keeplive connections