Tag: SL News

Of channels and restarts

Posted on by Inara Pey

Once upon a time server roll-outs for Second Life were handled in what, on the surface, would seem a fairly straightforward manner:

  • New code would be tested on the Beta Grid, with users reporting any bugs or issues to LL for fixing
  • When considered relatively stable, the code would be rolled out on a limited basis to the Main Grid (affecting around 20% of the grid in total) for further “testing”; if major problems were found, the limited roll-out (or “pilot”), would be rolled back
  • If considered stable, the code would be rolled out to the remaining 80% of the grid, generally around 24 hours after the pilot.

The system wasn’t flawless; the complexity of the server code meant that many small (and one would guess conflicting) updates would be “rolled-up” into a single release, often with unpredictable results, despite testing on the Beta Grid. This would result in what I call the “tidal effect”: a change would be rolled out as a pilot, then rolled back for fixing, then rolled out before being rolled back for further fixing, and then rolled out once more, then rolled out again to the entire Main Grid. Sometimes even then, it would go through one more rollback / rollout.

As we’re all only too aware, this approach meant fairly large and constant upheavals for just about everyone concerned, and the cause of much gnashing of teeth and dark mutterings towards Linden Lab.

To try and minimise the overall impact of server code updates and roll-outs, Linden Lab switched over to a “channel” system. Under this system, server code is operated across four channels: the Release Channel, with the latest “release version” of the server code (and supposedly the most stable), and three “Release Candidate” channels, code-named Blue Steel, Magnum and Le Tigre.

Each of the RC channels comprises about 10% of the total Main Grid, and is used to roll-out a “beta” of a specific server code package. This might be a series of bug fixes (e.g. specific SVC JIRA fixes), it might be a general maintenance release (e.g. security updates, etc.), or it may be related to a specific, on-going project (such as display Names, the “Fast Assets” project, the “Inventory Capabilities” project, and so on). Broadly speaking, specific projects tend to be rolled out through specific channels (The Inventory Capabilities project tends to rollout via Blue Steel, for example, as do changes related to the forthcoming arrival of Mesh) – although this is not a hard and fast rule. General maintenance releases, on the other hand, are distributed between all three channels, depending on which has the capacity at the time a release package is ready for beta testing.

So, at any one point in time, some 30% of the grid is hosting what is effectively “beta” software, but in very discrete “chunks”, so to speak, confined to known sets of simulators. The releases themselves are also smaller and more easily managed / identifiable, making everything that much easier to manage and, in theory at least, making issues that much easier to identify and correct.

Broadly speaking, this is how it works:

  • An update (be it bug fixes or whatever) is readied for release as a “beta”. If it is related to a specific project it may be targeted at a specific RC channel (Blue Steel, Magnum or le Tigre)
  •  On the Wednesday of each week, the Release Candidates for each channel are rolled out to their respective 10% of the Grid; if a specific channel doesn’t have a candidate waiting, this obviously, nothing is rolled out
  • Over the course of the next week, the candidate’s performance and impact on the Main Grid is monitored (and the channel servers may be subjected to numerous restarts. If the candidate proves particularly problematic, it may even be rolled back
  • If the candidate appears to be stable after 6 days, then it is (together with any candidates from the other two channels) rolled out to the entire Main Grid the following Tuesday
  • The cycle then repeats with the next RC in the channel dropping into its assigned servers on Wednesday.

If a specific RC causes problems, then the cycle for a specific channel may be broken for a week while the issue is worked on (for example, if a candidate on Le Tigre, say, proves that it is not ready for release as scheduled on a Wednesday, it will be “held over” for a week and made ready for release the next Wednesday).

There is one other channel worth mentioning that doesn’t get a lot of publicity: the “Snack” channel which handles releases related to (among other things) Mono-2 updates and various script monitoring tools. These are known to behave unpredictably, and so are initially rolled out to a very limited number of sims for testing. I understand that once tested, the fixes then go on for wider testing via (usually) Magnum prior to a full rollout.

The benefits of this system are obvious: if there is a major problem with a Release Candidate, it will only affect 10% of the Main Grid (rather than 20% with the old system); the releases are less complex, making it easier for the root cause of specific problems to be identified and corrected. Overall, the process means that there is less widespread upheaval across the Main Grid than tended to be the case with the old, larger-scale releases. There are many examples of these advantages; when a recent change impacted breedable horses, for example, it only affected a small percentage of horses on the grid (only those present on servers running the specific release channel software).

Of course, there are what appear to be downsides to the new system: the release channels (particularly, it would seem, Le Tigre), can be in a state of flux when problems do occur; and the weekly rollouts, with their need for sim restarts, on both Tuesday and Wednesday has been the topic of many a complaint. A minor irritant is the pop-up that comes up when moving between sims running different server releases, be they a release channel or the “full” release – it would be nice if these could be turned off by those who have no interest in what software is being run on a given simulator, just as other pop-ups can be user-disabled through the Viewer.

But, these grumbles aside, it has to be said the new system works. While it does cause a degree of pain for those “stuck” on simulators running one of the release channels, the vast majority of the grid has seen far less upset and upheaval when things have gone wrong. Certainly, the “tidal effect” of gird-wide rollouts/rollbacks has become largely a thing of the past, and while the rolling restarts associated with Tuesdays and Wednesdays might seem inconvenient when they begin, the truth is they’re probably less so than they were under the old system.

Viewer security exploit revealed

Posted on by Inara Pey

Nalates Urriah reports that Linden Lab have confirmed there is a security exploit involving a flaw in the Ogg Vorbis library could lead to Viewer crash issues. It’s not thought that the exploit can either perform privilege-escalation or arbitrary code-execution on users’ systems.

The flaw has been known about since 2009, but the exploit is fairly recent. Ogg files are in widespread use, so this is not an issue specific to the Viewer code. Linden lab has responded to the situation by issuing a patch and an advisory for all TPVs to recompile their binaries for all TPV viewers.

At the time from writing, updating executables for Kirstenlee’s Viewer (S21 7a) and the Firestorm Previews have been released.  Links for the Firestorm downloads (which do not appear to be available on the Phoenix website) are available as follows:

Note that all of the above three releases of Firestorm should be clean installations, not installed over any previous release (which should be removed first).

Other TPVs will doubtless follow, and users are advised to keep an eye on the various Viewer-related blogs and update as required.

Addendum May 16th

Phoenix have released an update that fixes this issue (and others). Find it here.

Local payments update

Posted on by Inara Pey

Frank Ambrose – FJ Linden – is an unsung hero of Linden Lab and Second Life. Since he’s been a part of Linden Lab, he has worked hard to communicate openly and directly with users in a manner that really cannot be faulted – and which should be taken as the standard be which others in the Lab should communicate.

Changes in the way non-US users can make payments has been the source of much controversy of late. Overseas Paypal payments were no longer acceptable, and the new local currency system has been less than confidence-inspiring.

These issues have lead to concerns among users as to what happens if payments fail – as has been the case – and Frank’s latest post on the subject is clear and concise on matters, and provides precisely the required level of reassurance on matters that is needed.

It’s also good to see that it actually shows up on the Featured News list on the Dashboard for once. Is this a sign that that particular problem is being fixed?

Another Alt detector surfaces

Posted on by Inara Pey

It seems there is another Alt detector on the loose. This one is a little less noxious than RedZone, and the speculation is that it is a variation on the previously banned Quickware scripts – although the author denies this.

As with all these things, it requires media streaming to be enabled on the target avatar’s Viewer – if they don’t have media enabled (or run Sione’s excellent Media Filter available in most Viewers) then the tool is a useless box of hot air. If one can talk favourably of the device, one has to say that the author is up front on this fact. Indeed, reading the advertising blurb gives reasons enough not to part with even L$250. I quote:

“* This method only works on Viewers having media enabled, there are no scripts or filters to stop it (NO THERE ARE NONE!) but simply deselecting media in preference is however an effective way to protect yourself from this and these kinds of devices. 
* This method is not 100 % reliable in any way and alot can go wrong. Actual detection means nothing more then that the target and detected share the same way into secondlife as in PROXY,Public Wifi-points (like mac Donald) etc. So take this into account when a positve match has been made !” [sic]

There is also mention of the detection method being flustered by dynamic IP addresses, which concludes, “It is important that the target ‘visits’ you often” in order to avoid the device being fooled by Dynamic IPs.

More interestingly, the author intimates that the tool is “safe” as it does not actually reveal IP addresses, nor does it collect or save them in any way. Technically, this would seem to bring the tool within the letter of the revised Community Standards, which restricts itself to the sharing of private information. However, whether this remains the case or not remains to be seen; if nothing else the tool does stand against Rod Humble’s own comments (to Dusan Writer) on people’s right to privacy where their alts are concerned – and it is really time that LL indicated they are prepared to stand behind Rod’s comments rather than staying silence on the matter of privacy, for reasons I’ve gone into elsewhere.

Overall, this new device is clearly intended to feed of Drama: it’s a one-avatar-per-box solution – that is, each purchased unit can only detect one designated target Avatar’s alts (allowing for all the caveats as to why it most likely won’t work).  This means that mass bans aren’t possible (unless one is willing to part with considerably more than L$250 in order to “protect” one’s venue).

But again, this isn’t really the point; if this tool is allowed to stay, there will doubtless be some 14-carat blockhead out there who will take it as a sign that they can make something even more intrusive, and we’ll be back on the merry-go-round once more.

Bouncing bewbs beget blog bit

Posted on by Inara Pey

Well, Enhanced Avatar Physics are finally formally released  – and blogged about by Samuel Linden, aided by Amanda Linden (honestly, the bouncy bits are that big they require a double team effort?!).

The news isn’t exactly new; people have been blogging and filming the new undulating body parts and having mild fits of hysterics playing with the sliders for a while now.

And truth be told, the enhanced physics are fun and add another level of realism to SL (and without the angst associated with the way bewbs bounce in viewers like Phoenix).

However, we’ve come through nigh-on a month of almost deafening silence from Linden Lab while a myriad of things bork, bomb and basically bugger-up (OK, not quite so alliterative, but you get the point). So you can understand it when people take a tongue-in-cheek line of reportage on the subject, or indeed, mix a little acidity into their view on the news, as Marx Dudek did (sorry; hard to avoid that particular alliterative bit) on Twitter:

What I like best about SL jiggly bits: My breasts bounce with just the right amount of realism while I hover after another failed teleport.

Yes, bouncy bits are fun – but unless this announcement marks a renewal of the Lab actively communicating with the community, people are going to remain pretty unimpressed, even if those of us blessed with them can have bits that wiggle and jiggle and bounce and flounce.

Sign-up pages and Basic mode of Viewer – working well

Posted on by Inara Pey

Figures collated by Tateru Nino (see below) show that the new Basic Mode of the Viewer have had a positive impact on user sign-ups, with a dramatic upward trend since its introduction at the end of March. The new sign-up pages, reported on yesterday, have had an even bigger impact, it would seem – and despite teething problems. The latter point was hinted at by Rodvik yesterday when he commented in a Tweet to myself:

 Thanks, yeah should be fixed quickly. Even with those the results have been amazing.

This is a positive thing, and shouldn’t be dismissed as it demonstrates that without additional marketing and promotion, SL can generate user interest.

What remains unclear at present is how well the new Basic mode does at retaining users – and as Rodvik rightly points out, it’s going to be a few weeks before the figures on user retention become clear. Certainly, I’d still be somewhat concerned, as the “new user experience” leaves a lot to be desired, as Theia Magic and others have commented upon.

Rodvik has also indicated that the Basic Mode is to be enhanced over time; as I’ve previously mentioned, while some additional functionality would benefit the Basic mode, there is a risk that add too much, and the line between Basic and Advanced becomes so blurred as to be non-existent. Given this, I still hold to my position that it is the Viewer HELP functions that need to be overhauled, rather than simply dropping more and more into Basic (unless said “more” is modular in format and can be activated at a time of the user’s own choosing).

I’m really hoping that ears at LL are receptive to the idea of a more integrated HELP for Basic / Advanced modes; it would get a lot of people “over the wall” in terms of transitioning from one to the other a lot quicker.

Beyond this, the issue of orienting users once they are in SL does remain a problem. Some have called for a reinstatement of the mentors; others have called for a revamp of the welcome islands, etc., in line with some of the privately run welcome areas. From Theia’s reports, it would seem a degree of policing of the LL-run welcome points needs attention. It is a little ironic that the Lab is paranoid about adult language getting into the forums and the like – but anyone with Voice enabled can be immediately verbally abused on arriving in-world for the first time.

Personally, I’m confident that Rodvik is more than aware of what needs to be done – as does Bagman Linden. That the Basic mode has had such a positive impact is good news, and I cannot see anything other than it being capitalised upon in the coming weeks and months as more of the initial user experience is brought into focus and revised / improved.

(with thanks to Tateru Nino by way of Rodvik Linden)