The server-side elements of the new avatar baking service (part of the Shining Project) continue to progress, alongside what Nyx Linden has referred to as, “Some pretty scary viewer re-architecting” which is being undertaken by the Lab in order to try to isolate some of the avatar baking aspects from the rest of the viewer code base. As such, he’s anticipating the eventual code merge to be “fairly significant” when it does happen, although at the time of commenting (the last TPV/Dev meeting, September 7th), the code was not in a condition where it could be used by any of the TPVs.
Bake fail: a familiar problem for many
Overall, the viewer elements of the project are the priority, with the aim remaining to get the viewer code completed first, and to make it available to TPVs so that test viewers can be built. This is likely to happen before the code is ready for any formal release, the aim being to allow TPV devs to carry out test merges and to let LL know anything else has been broken as a result of the changes made to the code in order for the new service to work.
Once the code is available for testing both within the viewer and on test regions on Aditi, Nyx will be looking for “as many people as possible to pile-on” and test the code in order to see how the service works and how it may break, so that by the time the viewer code is merged for release purposes, it will be as robust as possible.
Test Avatars
To assist with the work, Nyx is also looking for volunteers willing to take part in the initial round of testing using avatars wearing multi-layer outfits (e.g. outfits combining undershirt, shirt and jacket layers, etc; outfits using multiple elements of the same layer; outfits with system skirts and glitch pants, outfits using alpha and/or tattoo layers, and so on). Anyone interested in joining-in the testing should contact Nyx via e-mail or by sending him an in-world notecard, specifying the avatar name and details of the outfit itself. When volunteering, bear in mind that:
The outfit must be a Viewer 3 outfit, and your viewer must support the Current Outfits folder (which is used to drive the new service from the viewer end)
Testing will be on Aditi, and as such, the avatar and outfit must be available on Aditi. If necessary, you may need to update your Aditi inventory to make the outfit available.
Going Forward
The plan is still to have the new code support both the “current” method of avatar baking and the new baking service, until such time as the new service is fully deployed. This means that if a user is in a region that does not make use of the new baking service, avatar baking will continue to be handled using the viewer-side mechanism. However, if the user is on a region that utilises the new baking service, avatar baking will be handled through that, with a flag set via the region capabilities being used to distinguish whether or not the new service is available.
There is still no definitive time frame for the project because of the complexity involved in both developing the viewer code (which is currently using a branch of the SL development viewer, but will obviously be moved to whatever code release is current) and with the development of the new server-side service. As such, it is still liable to be at least another month or so before the code is ready for significant testing on Aditi.
This is a little overdue due to problems earlier in the week accessing the recording of the TPV/Dev meeting (which I was unable to attend in person). Thanks to Oz for sorting the problem out, allowing me to catch-up on overdue updates on this and JIRA matters etc.
On the 29th June, Linden Lab announced Project Shining, aimed at improving avatar and object streaming speeds. At the TPV/Developer meeting on Friday 13th July, the project was discussed in terms of how the various elements within it will affect Second Life viewers.
The following is a summary of that discussion, based on the recording of the meeting, and focused primarily on the viewer changes / updates that will be most directly seen / felt by the majority of users.
HTTP Library
Commencing at 22:30 into recording.
The aim of this project is to improve the underpinning HTTP messaging that is crucial to simulator / simulator and simulator / Viewer communications. Monty Linden is leading this project.
Key points:
LL will release a project viewer containing a new “wrapper” implemented around how data is handled and a new texture fetch library (see time frame comments at the end of this article)
Providing there are no major problems with the project viewer, the initial code release will move to a release version of the viewer
This will be followed by changes to group services and a “more ubiquitous” use of the library in the viewer – which is where Oz’s warning to TPV developers comes into play, as some services and the behaviours will start to change to improve throughput and reliability – and may even help improve the SL experience for those on older routers.
As a side note, some of this work has involved router testing aimed at determining what router hardware is compatible with Second Life. While it is hoped that work around the HTTP libraries will improve the SL experience for some using older router hardware as noted above, the tests have revealed that certain types of older router – Linksys WRT and Belkin G series routers were specifically named – are not compatible with running Second Life.
Avatar Baking
Bake fail: a familiar problem for many
Commencing at 32:38 into the recording.
The aim of this work (Project Sunshine) is to improve issues around avatar baking and to eliminate bake fail issues. It will primarily focus on moving the emphasis for the baking process from the viewer to a new Texture Compositing server. The viewer will retain some elements involved in avatar baking – the actual baking of the avatar shape (i.e. shape values and IDs) will still take place on the viewer side, for example.
Precisely how this new service will work on the server-side of things is yet to be fully determined by Linden Lab. However, work is progressing on the viewer side of the equation, with the current key points as follows:
The new service will use the Current Outfit folder to drive the new baking service
TPVs not currently supporting Current Outfits will have to implement it, otherwise they will effectively fail on avatar baking
The basic process will be that when it is time to send a rebake request (e.g. after a user has finished editing their appearance) the viewer must send a new message to the baking service which effectively says, “Look at the contents of my Current Outfit folder and give me back a new appearance based on that”
Viewers in general will have to support this new message that is sent to the service, and change how they perform the fetching of avatar textures; for the technically inclined, this will be HTTP without UDP fallback.
Currently, the plans is for LL to integrating the new way of doing avatar baking into their viewer code, which will be available for TPVs to integrate – although none of the Linden Lab 1.x code will be updated to support the new process, so this will effectively break their own Viewer 1.23.5, which currently is still in use within SL.
The viewer code will support both the “current” method of avatar baking (within the viewer itself) and the new baking service (using the Texture Compositing server) until the new service is fully rolled out across the grid. This means that if a user is in a region that does not make use of the new baking service, avatar baking will continue to be handled using the viewer-side mechanism we currently have. However, if the user is on a region that utilises the new baking service, avatar baking will be handled through that. The viewer will be able to recognise whether it is connected to a region supporting the “new” method through the region capabilities.
In order to ensure as smooth a transition to the new baking process as possible, LL are proposing a relatively long lead-in to the new service, making the code available well ahead of the new service being enabled, allowing TPVs to integrate it into experimental builds. The server-side changes will initially be implemented on a number of beta grid regions for testing with viewers there, prior to being scaled-up. The server changes will then be released onto the main grid in a controlled manner and then scaled up from there.
What Does This Mean for Users?
If all goes according to plan, and providing that you keep up-to-date with releases of your preferred viewer, this actually shouldn’t mean very much in real terms. There are however a number of things to be aware of:
If you use a viewer that is not updated to use the new code (i.e. the official viewer 1.23.5 or a viewer that is not updated to use Current Outfit folder and / or to support the new bake request message / HTTP texture fetch mechanism) OR you continue to use an old version of a viewer rather than updating, there will come a time when your avatar – and those around you – will not bake correctly
There are two issues that may occur during the transitional period when both the “current” and the “new” baking methods are in issue:
When teleporting or crossing between regions that use the different methodologies, users will experience their avatar rebaking, as the viewer will effectively be using two sets of data for the bake process
If there are two adjacent regions, one of which is uses the current avatar bake process and the other is using the “new” baking service viewers in one region will not be able to correctly resolve the textures of avatars in the other region
It is hoped that the transitional period where both methods of avatar baking are active will only last for about two weeks.
Object Caching and Interest Lists
Commencing at 57:25 into the recording.
When you enter a region at the moment, your viewer receives a huge amount of information on what requires updating, much of it relating to things you can’t even see from your position in the region. The data is received in no particular order, with the familiar result that things appear to rez in your view in a totally random order – quite often with the thing you actually want to see being one of the last to rez due to the mechanics of Sod’s Law. What’s more, if you have previously visited the region, the chances are that much of the information being sent to your viewer is already cached.
Object caching and interest list changes: easing the pain of random rezzing
The focus of this project is to optimise the data being sent to the viewer, information already cached on the viewer and the manner in which that data is used in order to ensure it is used more efficiently so that things rez both faster and in a more orderly manner than is currently the case.
At this point in time, this work is in a greater state of flux than the HTTP library and avatar bake projects. This is more a process of optimisation both on the server-side of things and within the viewer itself, rather than that of new functionality within the viewer per se. There are no general time frames for this work at present, but there will be updates once things become clearer as to how the optimisation is going to be addressed.
Time Frames
The precise timeframes for implementing these changes have yet to be properly defined. However, Oz Linden hopes that there will be at least a two month period between Linden Lab making the code for each of these project elements available for integration by TPV developers into their viewers and the point at which the Lab states the code must be in use.
At the moment it is likely that the HTTP library element of the project will but rolled-out first, although this is unlikely to be within the next two months, for the reason given above. Project Sunshine, dealing with avatar baking, will then follow after that – or although how soon after has yet to be determined; as described earlier in this article, this will be a very controlled roll-out. It is possible that the object caching / interest lists part of the project many not be rolled-out for another six months. However, timeframes are still in discussion within LL, so any of this may well change.
Expect updates on all three of these project elements as and when more information is supplied by Linden Lab.
Yesterday Linden Lab announced a major series of new initiatives aimed at improving the overall SL experience. The announcement came via a Tools and Technology blog post, which covers the initiatives in great detail. These focus on four main areas of activity, one of which is directly related to hardware and infrastructure, and the remaining three are focused on the platform itself and are grouped under the Shining project banner.
The hardware / infrastructure element of the work is described thus:
This year, Linden Lab is making the single largest capital investment in new server hardware upgrades in the history of the company. This new hardware will give residents better performance and more reliability. Additionally, we are converting from three co-locations to two co-locations. This will significantly reduce our inter-co-location latency and further enhance simulator performance.
The Shining project is something that is already known to many SL users – especially those who attend some of the User Group meetings. It is perhaps most famously associated with the Lab’s work on the Viewer rendering code, removing outdated functions and calls no longer supported in modern graphics systems (most notably Nvidia) and improving graphics handling overall. Shining has also been responsible for other incremental improvements to issues around streaming objects and avatars.
Under the new initiative, Shining is split into three core performance projects.
Bake fail: a familiar problem for many
Project Sunshine: One of the biggest complaints from users in SL is related to avatar rezzing. This can appear slow, and usually manifests in avatars remaining grey for periods of time, or in skin and system clothes remaining blurry (see right) – and at its worst, result in a user changing their avatar’s outfit – but others either seeing the avatar still dressed in the previous outfit or naked. Collectively, these issues are known as “bake fail” and are the result of the Viewer having to do all the compositing of avatar textures locally, then sending the results to the SL servers, which then send the information back to the simulator the avatar is in to be accessed by other Viewers in the same simulator.
Under Project Sunshine, to precis the blog post, much of this work is moved server-side, using a new, dedicated server, the Texture Compositing Server, which is separate to the simulator servers. This effectively allows all the “heavy” communications and calculations work relating to avatar texture calculations to performed within LL’s servers and across their own internal network, removing the reliance upon the Viewer and on Viewer / server communications which are outside of LL’s control.
Object Caching & Interest Lists: This is intended to directly address another common request from users: improving how the Viewer handles local object caching. This effectively means that once the Viewer has information relating to a specific region, and providing the information is still valid (i.e. there have been no changes to objects that the Viewer already has cached), then it will no longer need to re-obtain that information from the server. Only “new” or “changed” data needs to be streamed to the Viewer. This should mean that on entering a previously visited region, the Viewer should immediately be able to start rendering the scene (rather than requesting a download from the server), while simultaneously requesting any “updates” from the server through a comparison of UUID information and timestamps.
HTTP Library: The final aspect of Shining’s three-phase approach is to improve the underpinning HTTP messaging that are crucial to simulator / simulator and simulator / Viewer communications (and thus key to the other elements of Shining) through the implementation of “modern best practices in messaging, connection management, and error recovery”.
Overall, Shining will be tackling some of the major causes of Viewer-side lag and user frustration in dealing with avatar bake fail and the complexity and wastefulness of scene rendering that is encountered when moving around SL.
No definitive time frames for the improvements have been forthcoming with the announcements – and this is understandable; there’s a lot to be done and matters are complex enough that LL will want to proceed with minimal disruption to the grid and to users. Doubtless, more information will be made available as becomes known through the LL forums and (possibly more particularly) via the relevant User Groups.
Inara Pey: Living in a Modemworld 