While reading the transcript of the Simulator User Group meeting of Tuesday 25th September as a part of preparing my last SL projects update, I came across an interesting exchange on the subject of large regions –  megaregions in OpenSim parlance – which gives some insight into the broad level of thinking about the platform that goes on within the Lab.

For those unfamiliar with the concept, a megaregion is essentially a number of standard 256×256 metre regions stitched together to present what appears to be a single large region. These are generally presented in terms of areas equivalent to 2×2 regions (i.e. 4 region in total) or 3×3 (equivalent to nine regions) and so on.

Megaregions have been available within the OpenSim environment for the last few years, and are seen as means of providing far more space free from the terrors of region crossings, greatly facilitating a range of activities – flying, sailing, vehicle racing, etc., – although there are some limitations with them at present, which can make working with them difficult (parcel media tends to be restricted to the South-west corner “region” of a megaregion, for example, and elements such as terrain textures cannot currently be easily edited).

Second Life is very much geared to the 256×256 metre region, so it was surprising to come across a discussion on large regions in SL – and to learn that Linden Lab have in fact looked at them in some detail. The revelation came in a comment from Simon Linden, “Yeah, big regions have been a pet project of mine … unfortunately it’s an incredible hassle to get right,” he goes on to say, a short time later:

“I’ve spent some serious time looking at large regions … it’s a huge project to do it right, involving a bunch of messaging changes to the viewer (like layer data, object positions, etc), region-to-region communication (all the neighbours) our back-end (the grid layout itself) … it touches almost everything in some way, which is why we’re where we are today 😛

Simon also indicated that he felt an ideal size for large regions – were they ever to happen – would be to a scale of 1 km on a side, rather than  1024m on a side (as would be the case if large regions were somehow “scaled up” from the current region size, as with OpenSim). However, this would mean breaking away from the current power of 2 approach to building Second Life, and might lead to position translation issues (as in translating the position known in one region to the relative position in a neighboring region), although Andrew Linden felt this might actually be easier to handle this in 1k blocks between neighbouring regions, rather than relaying on power of 2. When asked as to what would happen to the 24 metres per side which would be lost in scaling to 1000x1000m, rather than 1024×1024, Andrew suggested (semi-jokingly) that they’d be lost “To … boundary conditions.”

Were any change in region sizes to be undertaken, they would not be limited to just the simulator / server-side of things. The viewer itself is predicated on the power of 2 approach, being specifically geared to handling regions of 256m on a side (hence why megaregions in OpenSim have some limitations in terms of editing, etc.). So for large regions to work properly, it is likely that substantial changes would have to be made to the viewer – which even with the best will in the world, isn’t something which is going to happen any time soon, even were LL pondering looking beyond the theoreticals of large regions.

Nevertheless, the fact that the matter has been – and might still be – something some in Linden Lab are actively looking at, even at only the conceptual level, is interesting, and does tend to demonstrate that LL do think about the platform in somewhat radical ways.

The dust is slowly settling from the recent announcement vis the effective closure of the Public JIRA for bug and issue reporting and the implementation of the simplified Bug Tracker approach and associated changes.

Comments passed from front-line staff by Linden Lab make it reasonably clear that the new approach to bug reporting and management has impacted more than just those users who have in the past been actively and positively engaged in the Lab’s JIRA; the Lab itself is undergoing something of a shift in how issues are handled, and that is it likely to be a few weeks before matters settle down internally.

The Lab is also adamant that the overall aim of the change is to try an improve the utility of the bug reporting and management process from their own perspective – part of which was to eliminate the issue of having the JIRA used either as a forum for discussion and / or for posting irrelevant / angry statements, neither of which were seen as assisting the process of problem management and issue resolution. However, there has been an acknowledgement in some quarters as to whether or not the new system will increase or decrease the effectiveness of bug tracking  / management over time is an open question at the Lab, and that depending upon how the new system is seen to work over the next weeks / months, further changes may be made.

“JIRA Support Groups”

During the TPV/Dev meeting on September 7th, Oz Linden indicated that there are two “user groups” which are being established in relation to the new changes, and which the Lab will use to allow those residents with a demonstrable need to access a JIRA system and who are known to do so “responsibly” to have greater access to the new system.

Commenting after it became apparent during the meeting that some in attendance already had greater access to LL’s JIRA than others (including the ability to still comment on JIRA items), Oz said:

It should be noted that not all of you have exactly the same privileges. As part of this change [to the JIRA system] I created some access groups that do have somewhat deeper access … I haven’t actually figured out exactly what got set-up in the end … so be a little careful about asserting that, “Anyone can do such-and-such”, because if you’re in the active contributors’ group or the support helpers’ group, you have privileges other people don’t have … As I said, these changes have only been in effect less than 24 hours now [at the time of the meeting] … because there are a couple of levels of indirection involved, it’s not trivial to figure out what privileges a given person has – which is weird, but there you go … So, I have put in place a mechanism that I hope will make it easier for those of you who are actively collaborating with us on making the world better to continue doing so. It will probably take some time for all the bugs in that accommodation to be worked out.

Later in the meeting, he indicated one of these two groups, the “active contributors’ group” is being aimed towards the likes of TPV developers and those who have contributed to Second Life in terms of code and fixes, etc., in order to try to ensure they continue to have access to the new system which is beneficial to them (and more particularly, to LL) in order to better resolve bugs.

Similarly the “support helpers’ group” will be overseen by Alexa Linden and will comprise those who have demonstrated their value in assisting with the broader triage process (such as identifying duplicate issues, recognising where short-term workarounds for problems may exist, etc.).

Both groups were referred to as having greater ability to search reports in the new system, although the precise function and capabilities of these groups is liable to mature alongside the new system. While some people have already been added to the groups, this has been done as something of a “first pass” and appears to have been based upon first-hand knowledge of those involved. How additional people will be added to each of the groups is not entirely clear, although it is evident that in order to qualify for consideration, an individual must have a track record of positive and beneficial engagement in the JIRA process to triage and / or resolve issues.

Also during the meeting, Oz encouraged TPV developers who are concerned about the negative impact of the change and who have “Legitimate use cases that serve the needs of Second Life in general and Linden Lab in particular,” which may not be met by the new system, to write them up “In non-emotive form, … [but] in terms of how they are useful to Second Life residents and how they provide utility to Linden Lab … a calm exposition of the value to Linden Lab of doing something different would be.”

Forum Discussion Option

The JIRA situation was also raised at the Simulator User Group Meeting, also held on September 7th, Simon Linden put forward a suggestion that perhaps the forums could be used in some capacity. He was encouraged by those attending the meeting to pass the idea back to the Lab itself, with Toysoldier Thor suggesting a new Forum category of “Post-JIRA Forums” to facilitate general discussions. During the Content Creation User Group meeting held on the 10th September, Alexa Linden further indicated that the possible use of the forums was being considered.

Going Forward

The debate on the positive / negative aspects of this change are liable to continue for some time to come. That steps were taken to create two new “JIRA support groups” ahead of the launch of the new system tends to demonstrate that some within LL were not blind to the part played by users in the overall management and resolution of bugs. The hope appears to be that these new groups will offset the more negative aspects (lack of access, ability to contribute, etc.), presented with the launch of the new system.

Whether this proves to be the case will come down to how effectively the groups are managed, the level of access those within the groups are given, and whether or not the new system itself achieves the level of improved utility in the reporting, triaging and resolution of bugs the Lab hopes will be the case. Currently, it would appear that none of this is liable to be objectively known for the next several months.

Related Links

• Linden Lab JIRA changes announcement
• My initial report on the change
• TPV/Dev meeting recording, September 7th (core JIRA discussion starts at some 30 minutes in)
• Simulator User Group Meeting archive, September 7th

Linden Lab today reported that they’ve effectively closed the Public JIRA system to users, and are launching a new “bug reporting project”.

The announcement, made in the Technology blog, reads:

User-submitted bug reports help improve the Second Life experience for all Residents, so we greatly appreciate all of you who take the time to provide this invaluable information to us. 

Because we want to make it even easier to report bugs, today we are making some changes that will streamline the bug reporting process, allowing us to more quickly collect information and respond to issues.

Following is a summary of the JIRA changes:

• All bugs should now be filed in the new BUG project, using the more streamlined submission form.
• Second Life users will only see their own reported issues.  When a Bug reaches the “Been Triaged” status, they will no longer be able to add comments to their issue.
• Once a Bug reaches the “Accepted” or “Closed” status, it will not be updated. You can watch the Release Notes to see when and if a fix has been released for your issue.
• Existing JIRAs will remain publicly visible. We will continue to review and work through these.

To those of you who have taken the time to alert us to bugs and provided the information we need to fix them — thank you! We hope that you will continue to help us improve Second Life, and this new process should make it easier for all of us. Ideas about how we can continue to improve the bug reporting process can be shared here.

For more information, visit:
How to report a BUG (Knowledge Base Article): 
Bug Tracker (wiki page):
Bug Tracker Status/Resolutions (wiki page)

As a part of this change the public JIRA is still browsable, but it appears the ability to comment on specific JIRA items has been turned off.

It’s hard to fathom why this has been done – and the stated reason actually makes little sense. If nothing else, the fact that users can only see the bugs they report will inevitably means that the system is liable to get flooded with duplicate entries  – far more so  than is was the case with the JIRA system. Beyond this are other aspects which seem to make this move counter-productive:

• Users are often a part of the triage process. They can confirm when and how issues are occurring; they can test different hardware and different viewer options and ascertain if the problem is at all localised, or possible an artefact unique to the reporter’s system
• Developers can similarly – and vastly – help the triage / resolution process, bringing their own knowledge and skills to bear on user-reported problems
• Both users and TPV developers can speed the process on duplicate JIRA identification and cross-referencing, reducing the amount of work LL have to initially undertake.

All this move appears to do is further break another means of productive collaboration between Linden Labs and TPV developers / the user community, leaving everyone the worse off, and that in itself is hardly positive.

While there has been frustration within LL – and among those who do invest time and effort in trying to help LL deal with raised JIRAs – over the amount of (often pointless) feedback,  bickering than can occur with a particularly emotive JIRA (comments like THIS IS BAD!!!!!!! FIX IT NOW!!!!!!! certainly don’t help anyone), this move can hardly be called a proportional response to preventing such problems.

Unless there is more to come, such as TPVs at least being allowed to engage in the bug / issue reporting / triage / resolution process, there is potentially only one adjective which some might opt to apply to this move.

Asinine.

As mentioned in the TPV/Developer meeting of the 24th August, Oz Linden has been taking steps to try an improve how issues are addressed by the company’s support teams when dealing providing support to users who are using a TPV as their viewer of choice.

That TPVs are collectively more popular than the official SL viewer is not that surprising. However, a lot of people still turn to Linden Lab for help when they encounter issues. As a result of this, LL have come in for criticism as to how they handle users who report that they are using TPVs, and it is this that has prompted Oz to try to improve how matters can be handled and addressed.

Identifying the Problem

The first part of dealing with any problem is correctly diagnosing whether it is in fact viewer-related or server-related. This isn’t as easy as it sounds because there are many parts of SL where the problem could reside either within the viewer or on the server-side of things (inventory issues being a good example) – hence why LL often get the call when things go wrong.

Because of this complexity, and in order to help improve the initial viewer issue / server issue diagnosis, Oz is working with LL’s support teams to put together a better set of heuristics for use in support staff training and guidance in identifying where a particular problem may reside. To help with this work, he has asked the TPVs supply lists of issues they have encountered which they know are not viewer issues, and how to recognise them. These lists can then be added to the information supplied to LL support staff to both speed the initial diagnosis of a problem and reduce the chances of a problem being mis-diagnosed from the outset.

It’s a Viewer Problem – But Can it be Reproduced on the LL Viewer?

When it comes to trying to resolve what appears to be a viewer issue, LL support staff will ask a) whether the user is using the official LL viewer; and b) if they have tried to reproduce the issue using the official LL viewer. These questions are often taken to mean LL’s support staff “do not want to help” with the problem if it appears to be TPV related.

However, this is not the case; the question is a perfectly valid part of trying diagnose a problem because:

• If the problem can be reproduced using the official viewer, there is a chance support staff may be able to provide SL-viewer based assistance to resolve the issue
• If the problem cannot be reproduced on the official viewer, then it at least helps point to the problem potentially being related to the TPV itself.

Obviously, if the problem does appear to be viewer-related but only manifests in a TPV, LL’s support personnel are unlikely to be able to give detailed help (simply because it is unfair to expect LL’s support personnel to be intimately versed in how to resolve issues occurring with all of the TPVs used to access SL). As such, they are going to pass the matter back to the user. When this happens, it can lead to frustrations and a feeling that LL “aren’t interested” in solving the problem.

To avoid this in the future, Oz is working with TPVs to ensure LL’s support staff are better placed to provide onward guidance rather than leaving users feeling they “don’t want to help”. This is being done by each TPV listed in the TPV Directory being asked to:

• Add the details of any in-world support group(s) they operate to their Directory listing if they haven’t already done so
• Use a new field in the Directory to give details of any additional locations where help on a specific TPV might be obtained (e.g. a website, a support forum, etc.)

Thus, should an issue appear to be related to a specific viewer which LL staff cannot help resolve, they will at least be able to point the user concerned in one or more directions where they can receive more focused assistance in order to resolve the problem.

Asking People to Complete the Survey

During the discussion, Oz reiterated that every support issue dealt with by LL staff should trigger a follow-up e-mail to the user concerned. While this might not happen until up to four days after the event itself, the e-mail does include a customer satisfaction survey. This is important for two reasons:

• All survey responses are reviewed by a Linden Lab staffer; they are not farmed out to a third-party survey company or ignored or handled by an automated process
• They are seen as a primary mechanism for determining how well support is identifying and dealing with issues to the satisfaction of LL’s users.

As such, Oz emphasised the importance for feedback to be given, particularly where there is strong evidence to show that support have failed to provide the correct assistance. While completing the survey may not help in resolving the issue itself, it may help pin-point errors within the support process, particularly if a number of surveys are received highlighting the same fault.

The current process by which support issues – particularly those with TPV problems reported to LL – are handled doesn’t always run smoothly, and there are times when issues do get mis-directed. However, Oz’s response to concerns raised during recent TPV developer meetings demonstrates that steps are being taken to address them. It has been suggested that LL post a blog entry on the initiatives explained here (particularly on the need for TPV users to understand why LL do ask about reproducing issues encountered using the official viewer). In lieu of that happening, I hope this piece will serve as an informational.