Lab issues further CDN update – more improvements coming

On November 1st, 2014, the Lab blogged about improvements seen from their side of things as a result of CDN support deployment.  At the time the updates were being issued, the Lab was also asking for feedback from users as to how things were going for them.

As a result of this request for feedback, the Lab issued a further update on the improvements on Friday, November 7th,, and it is a tale of two halves.

The first part of the blog post re-states the core benefits that have been seen as a result of the CDN deployment for mesh and texture data, which is again split into two key areas: a considerable reduction in the load on some key systems on the simulator hosts, and a big performance improvement in texture and mesh data loading, resulting in users seeing faster rez times in new areas they’re visiting.

In the graphic released with the Lab's November 7th update on the CDN deployment,
From: An update on the CDN project, linden Lab, November 7th, 2014

However, the experience of some users hasn’t been so good, as reported in the forum thread, and it could not be put down to matters of distance from the CDN nodes vs. the Lab’s simulators, or to people experiencing slower load times as a result of being the very first to enter a region which had not been cached at the local CDN node.

This feedback encouraged the Lab into further investigation and data-gathering of specific situations, allowing them to engage with CDN supplier Highwinds in order to try to determine possible reasons for the poorer experiences. The second part of the blog post notes the outcome of these efforts:

We believe that the problems are the result of a combination of the considerable additional load we added to the CDN, and a coincidental additional large load on the CDN from another source. Exacerbating matters, flaws in both our viewer code and the CDN caused recovery from these load spikes to be much slower than it should have been. We are working with our CDN provider to increase capacity and to configure the CDN so that Second Life data availability will not be as affected by outside load. We are also making changes to our code and in the CDN to make recovery quicker and more robust.

The blog post also points out some of the risks involved when trying to deploy large-scale changes to a complex and dynamic environment such as Second Life:

Making any change to a system at the scale of Second Life has some element of unavoidable risk; no matter how carefully we simulate and test in advance, once you deploy at scale in live systems there’s always something to be learned. This change has had some problems for a small percentage of users; unfortunately, for those users the problems were quite serious for at least part of the time.

The post concludes by thanking all those who contributed to helping the Lab understand the nature of the problems being experienced and in taking the time to help provide data on their particular circumstances which helped with further investigations, and with a note that it is hoped that the changes that are to be made as a result of this work will reduce such problems, allowing more people to enjoy the benefits offered through the use of the CDN for asset data delivery.

3 thoughts on “Lab issues further CDN update – more improvements coming

  1. I was one who commented in the negative. For example, I go to a friend’s castle every day as I help him out there. While it has always taken me a minute to two to see the entire room rezzed to me, for the past while, this has changed for the worse. Now, when I land, the entire room is grey and it takes up to two minutes just for the grey to fade and show anything. However, now I have to wait for the objects to change from spheres, triangles, etc. to the actual object fully rezzed. I’m waiting for as long as 15 minutes now. Now, before you ask, we have not changed anything at the castle to impact performance like this AND I’m seeing the same thing at a lot of sims I go to. Saturday and Sunday nights are generally pretty dead on SL and I go shopping where sometimes I’m either the only person on the sim or part of a handful and I’m seeing the same thing. Nothing has changed in my home network and I reboot my modem and router weekly to be assured of top performance. This only just began happening in the past month or so.

    Like

  2. Well thuis weekend i did noticed a lot of improvements, mainly when crossing sims in mainland, all seems much smoother now even if some weird old problems still arise:
    Camera changes angle on a less smooth cross sim when driving a bike in mainland,
    If zooming to far from the vehicle (sailing boat) and let it cross to another sim, when coming back the avatar is still on the previous region (if i manage to see the boat i double click and teleport to the boat region and i mange to get back, even with the pose screwed). Still these issues already happened prior to cdn so i know they are not related.
    On close sims all really rezz much slower the 1st time but then coming back is wonderful as it rezzes in less then a few sec.
    On mainland, that is not so obvious, still i did cross more then 200 diff regions just on sunday and did not had a single crash, using a low draw distance (248m) i noticed that the trees where rezzing faster even if they are still the texture that takes longer to rezz everywhere (LL trees).
    To note that i was on heavy traffic areas (Blake sea near san catalina) and still manage to do a race with quite a few sailing boats (OVER 10) and nobody had issues.
    So, even if im not using latest ll code (Ukando those not have the latest pipeline yet), i can really say now i do see improvements.
    My soul mate, on East Usa coast, with same hardware, viewer and sort of similar connection, had a few more crashes, but she also saw the rezzing time improvement.
    What we noticed was some other avatars are at live shows with missing parts and all them where using the latest ll official viewer and for them , all was fine, a relog show them that indeed they had missing attachment parts (mostly feet alphas).
    Still if i understood, LL already made a few corrections so i do hope more viewers will move to the pipepline (but firestorm lol).
    .

    Like

Comments are closed.